Chapter 5 Conceptualization, Operationalization, and Measurement

4. Operationalization choicesa. Range of variationb. Variations between the extremesc. A note on dimensions

d. Defining variables and attributese. Levels of measurement

f. Single or multiple indicatorsg. Some illustrations of operationalization choicesh. Operationalization goes on and on

5. Criteria of measurement qualitya. Precision and accuracyb. Reliabilityc. Validityd. Who decides what's valid?

e. Tension between reliability and validity

6. The ethics of measurement

SUMMARY

Social scientists believe that anything that is real can be measured. This is accomplished through thetwin processes of conceptualization and operationalization. Conceptualization involves specifyingwhat is meant by a concept, while operationalization yields the specific measures. Social scientistsalso commonly distinguish measurement from observation. Observation is a more casual and passiveactivity while measurement involves careful and deliberate observations for purposes of describingobjects and events in terms ofthe attributes of a variable. Most of the variables social scientists studydo not actually exist in the same way that the book you are reading exists. Instead, they are made upand seldom have a single, unambiguous meaning.

Conception reflects the mental images we have of something. Coming to an agreement about what

terms mean reflects conceptualization and results in a conceQt. Scientists measure three things. First,they measure direct observables-things we can observe simply and directly. Second, they measure

indirect observables-Iess direct methods used to provide inferences about concepts. Third, theymeasure constructs-theoretical creations based on observations but that cannot be observed directly

or indirectly. In short, concepts are constructs derived by mutual agreement from conceptions. Wesometimes err by believing that the terms for constructs have intrinsic meaning, an error known asreification.

Conceptualization is the process through which we specify what we mean when we use particularterms. Concepts themselves are not real or measurable; the indicators of a concept provide thisfunction. Indicators are signs of the presence or absence of whatever concept we are stud ing.Seldom can a single in lcator capture t e full richness 0 meanmg m most concepts. As a result,

83

Chapter 5 Conceptualization, Operationalization, and Measurement

social scientists develop multiple indicators arranged around s ecifiable aspects of a concept, knownas dimensions. Multiple dimensions and mu IP e In lCators contn u e 0 more sop isticatedundefSThndmg of our concepts.

To the extent that multiple, interchangeable indicators produce similar findings, researchers canreach similar conclusions about general concepts, even ifthey disagree about definitions. This stance

is based on the assumption that if di:ff~r~e_nt~in~d_i.~c_at_o_rs_al_l_re_p_r_es_e_n_t_th_e_s_am_e_c_o_n_c~ep,_t_,t_h_en_al_l_of_t_h_em_will behave the same way that the concept would behave if it were real and observable;

Three types of definitions contribute to conceptual order. Real definitions reflect the essential nature

of a conce2.t but are not very useful in the measurement process. Conceptualization instead dependson nominal and operational definitions. A nominal definition is one that is assigned to a term in a

given study, and an operationa~_e_fi_ni..,..,ti,.....-~_il_s_t_h_e_sp~e_c_ifi_l_c_op_er_a_ti_o_~_i_nv_o_l_v~~g-theconcept. These elements of conceptualization underscore the process nature of this step and produceconceptual order so that communication and measurement can proceed. Interestingly, definitions aremore problematic for descriptive research than for explanatory research because the goal of descrip­tive research is to describe something accurately.

Social scientists have a variety of options available regarding the measurement process. For one, theymust decide on the range of variation appropriate for a given concept. This choice is influenced bythe nature of the study and the expected distribution of attributes among the subjects under study.

Another choice pertains to how fine the distinctions will be among the vario~s possible attriE~of a given variable. This choice is also influenced by the purpose of the study, and social scientistsgenerally err on th~ side of gathering too much detail rather than too little when they are unsure abouthow to proceed.

Many times the selection of specific indicators fora variable is quite straightforward because such

variables have obvious single indicators (such as age). Other variables are better operationalize~through composite measures that combine a number of indicators reflecting separate dimensions(such as religiosity). In either case, attributes for a variable need to be both exhaustive (a complete~ of attributes) and mutually exclusive (no overlapping attributes).

Different variables may represent different levels of measurement. Nominal measures employ•••• ~A~ •.•••• ~

n~assification purposes only; their attributes have oJ}lythe characteristics of exhall~tive-ness and mutual exclusiveness. Variables whose attributes may be logicallv rank -ordered are ordinal1l1~su~; the numbers employed reflect relatively more or less of the variable being measured. Theactual distance separating attributes in interval measures can be described as being equal, but interval

measures have no absolute zero. Ratio measures do have an absolute zero value, e~~~thec~lation of r;ic;'s. It is important to identify the l~ of measurem~ the variab es beca~eanalytical techniques vary according to the level of measurement of a variable.

Social scientists apply several criteria to enhance measurement quality. They strive to developmeasures that are both precise and accurate. They pay particular attention to reliability (consistencJ0and validity (actually measuring the concept of interes,t). Problems with botlia.ppear repeatedly insocial researcn. Reliability can be enhanced by asking people only things they are likely to know,hy

84

-------------.-.--.-----.' ..~:::::::::~~~=__J

Chapter 5 Conceptualization, Operationalization, and Measurement

asking the same information more than once (test-retest method), by randomly assigning items totwo sets-and comparing the responses in the two sets (split-half method), and by using measures

known for their reliability. ~_onsistent application of the research design is also mandatory, what isknown as research worker reliability.

Validity can be enhanced through establishing face validity, whereby measures are compared withcommon agreements about a concept and with a logical understanding of a concept. Criterion-relatedvalidity (or predictive validity) is based on an external criterion. Construct validity is based on theway a measure relates to other variables within a system oftheoretical relationships. Finally, co~tyalidity refers to the extent to which a measure covers the range of meanings included in a concept.Social researchers need to consider both their colleagues and their subjects as sources of agreementon the measurement of concepts. Researchers also need to be sensitive to the trade-offs betweenvalidity and reliability in their quest for sound measurements. Note too that it is unethical to deliber­

ately attempt to slant your results by using a biased definition of your concept.

TERMS

1. accuracy (156) 16.2. concept (133)'e '7,-, \"- "'.\ ~ Q'A.LC>!"'\\JC\\,'t... 17.3. conception(133)' :"}\(---"71 18.4. conceptualization (136) 19.5 (135) 0 \ a\feC\ Q' u7- 20· constructs .'" . ,0 ,\.\ ;:N)"pV •\ \'\d-\ <..c-\

6. 'construct validity (161)\<><"c='re\oLOV\ 21.7. content validity (16IrD~,,\<- o( ~( ••,'~:722 .

.........-Fc>_&.,...J C

8. criterion-related validity (161):c~~~\'c~.·\23.9 d' . (137f''\'<'c·,.f-'c...~ l,' ~c'\"''''24· ImenSlon F~"i(~A; C> .>.-- •t...c-ccv

10. direct observables (134) . \\ ''<10\1:.. 25.11. maustive (149)t\\ In ~""Ii:.... U. ~ 26.12 ~ rd' (160)'~~-U"' ,c..-.?c. '-'1D0\- 27· lace va 1 1ty lOt 0 J "-I"'C'~" Ie,<::tV"l •

13. indicators (136t':;~-y':;rvY~:c'~:cC(~\- 28.14. indirect observables (135}=;,v'Q-\\.((. 29.15. interchangeabilty of indicators (139) 30.{J c\.", ,(~"'-ee.-.c......,),-- ,~~c:."---'~«)~(:y\ \.",~...\J,1.•.•,~.f1" .•••/(:" _"",., c. t (" .....•,,(''\7' rO .•.../\o. '(')<"'_

"0 \J .., <- {·V ,,0. .." .

MATCHING

85

Chapter 7 The Logic of Sampling

3. The theory and logic of probability samplinga. Conscious and unconscious sampling biasb. Representativeness and probability of selectionc. Random selection

d. Probability theory, sampling distributions, and estimates of sampling error

4. Populations and sampling framesa. Review of populations and sampling frames

5. Types of sampling designsa. Simple random samplingb. Systematic samplingc. Stratified samplingd. Implicit stratification in systematic samplinge. Illustration: Sampling university studentsf. Sample modification

6. Multistage cluster samplinga. Multistage designs and sampling errorb. Stratification in multistage cluster samplingc. Probability proportionate to size (PPS) samplingd. Disproportionate sampling and weighting

7. Probability sampling in review

8. The ethics of sampling

SUMMARY

Having specified what is to be studied, the next task in the research process is selecting a sample.Sampling affords the social scientist the capability of describing a larger population based on onlya selected ortion of at 0 ulation. Two b.!Qa~HY-l2esof sampling methods are probabilit):. samplingand nonprobability ampling. Some sampling methods are more accurate than others; the earlyfailures and the more recent successes of political pollsters illustrate the refinement of samplingprocedures and the increasing reliance on probability methods.

Many research situations preclude the use of probability sampling, especially when it is very difficultor impossible to create lists of the elements to be sample. In these situations, researchers employnonprobability sampling strategies. Reliance on available subjects is used often. but is a very~method because it limits the generafuiiblTityofthe results. Purposive (or judgmental) samplingoccurs when a researcher selects a sample based on his or her own knowl~g~ of the population, itselements, or the nature of the research study. Snowball sampling is often used in qualitative fieldresearch and is particularly useful when the members of a special population are difficult to locate.

123

Chapter 7 The Logic of Sampling

It involves collecting data on those members of the target population one is able to find and then

~sking these respondents to provide information needed to locate other memhers of that populatioE'

Quota sampling strives to attain representativeness by constructing a matrix representing one or mo!echaracteristics of the target population, and then collecting data from persons having the requiredcharacteristics of a given cell. But it is often difficult to secure adequate informa1im!. to createaccurate quota frames (the proportions that different cells represent), and b~Qrproblem. As a final nonprobability design"field researchers may sometimes use informants, who aremembers of the group under study, tQJ;1rovideinformation about the group itself rather than aboutthemselves. It is best to select informants who adequately represent the group under study, but realizethat informants' willingness to respond to outsiders may reflect their marginal status within thegroup.

Because humans are very heterogeneous, it is important to select samples that adequately representthe population under study. Probability sampling allows the selection of samples not subj~9tto_theresearcher' sociology biases. A sampleis considered representative of the population from which it

is selected if the agwgale.....Q@Iacteristics of the sample closely' approximat~~e same aggreg~techaracteristics in the population. Hence, probability sampling occurs when every element has anequal chance of being selected (an EPSEM sample). Realize that probability samples rarely repre­sent perfectly the populations from which they are drawn. However, probability samples are morerepresentative than other types of samples, and probabili!y theory permits a statistical estimate ofthe accuracy or representativeness of a sample.

Probability sampling requires familiarity with several components. An element is that unit aboutvffiich information is collected, and it is typically the unit of analysis ofthe study. There are twolevels to~ a sample can be generalized. A population is a theoretically specified aggregationof study elements. A study population is the aggregation of elements from which the sample isactually selected. lp. random selection, each element has an equal chance of se1e.c.tiollindependentof any other event in the selection process. Random selection enables us to select elements in such

a way that descriptions of those elements portrays accurately the total population. A sampling unitis that element or set of elements considered for selection in some stage of sampling.

Two reasons for using random selection methods are that 1) this procedure serves as a check_onconscious or unconscious bias on the part of the researcher, and 2) this procedure is based onprobability theory, which yields the basis for estimating population characteristics as well ~estimating the accuracy of samples. Probability theory is a branch of mathematics that provides thetools researchers need to devise sampling techniques that yield representative samples and to analyzethe results of theiLsampling statistically. Probability theory provides the basis for estimating theparameters of a population. A parameter is the summary description of a given variable in a popula-.tiQ!1.

The calculation and interpretation of sampling error lies at the root of probability sampling theoryand is grounded in the principle ofthe sampling distribution. The sampling distribution dictates thatsample statistics will be normally distributed around the population mean. Probability theory alsoprovides formulas for estimating how closely the sample statistics are~clustered around the true

population va~e. This is accomplished through the ll;e of confidence levels (how confident we are

124

Chapter 7 The Logic of Sampling

that our sample estimate is within a set number of sampling errors of the population value) andconfidence intervals (the range between the upper and lower values for a given level of confidence).A~esult.JY~ estimate a population parameter and the expected degree of error on the basis ofone sample drawn..fmm a population. The logic of confidence levels and confidence intervals alsoprovides the basis for determining the appropriate sample size for a study. Interestingly, th.~12opula­tion size is almost irrelevant for determining the accuracy' of sample estimates. Unless a samplerepresents five percent or more of the population it is drawn from, that proportion is irrelevant.

However, probability theory operates on a number of assumptions that are seldom met in real-lifesurvey situations, such as: an infinitely large population, an infinite number of samples, and sam­pling with replacement. Probability theory is useful only to the extent that the researcher can actuallyselect aprobability sample. Researchers sometimes overestimate the precision of estimates producedby using probability theory.

A sampling frame is the list of elements from which a probability sample is selected. Difficultiesin securing an adequate and accurate sampling frame put constraints on the use of probability theory.Remember that findings based on a sample can be taken as representative only of the aggregationof elements that compose the sampling frame. Sampling frames do not always truly include all thedements that their names might imply. All elements in a sampling frame must have equal representa­tiveness in order for the results to be generalized even to the population composing the samplingframe.

Simp-Ierandom samJ2ling is generally assumed in probability applications. This strategy involvesassigning a number to each element and using a t~ble of random numbers to select elements fOJJhe

s~ple. But simple random sampling is seldom used because it is not generally feasible and it maynot be_the most accurate metOOd. Systematic sampling is generally preferred over simple randomsampling because of simplicity and because it can be more accurate than simple random sampling.O_ncethe sampling ratio (the proportion of the population) is determined, the researcher simplys~j:sJh~J~l~ments corresponding to the sampling interval (the distance between elements selected),with the first element selected with a table ofrandom numbers. But researchers must be on guard

for p~iodicity, a cyclical pattern that coincides with the sampling interval.

Stratification may be used with both these strategies, and it increases representativeness by first

o!.ganizing the sampling frame into homogeneous groups reflecting variables that may be related tothe variables under study. Such homogeneous groupings reduce sampling error. Implicit stratificationoccurs when a sampling frame is organized by factors relevant to the study.

Multistage cluster sampling is most useful when no master list exists to provide a sampling frame.The researcher employs multiple sampling units such that groups of elements are sampled~tdifferent stages; the element is the final stage. But this design .Qroduces higher sampling errorsbecause each stage yields additional sampling error. A general guideline is to maximize the numberof clusters selected while decreasing the number of elements selected per cluster, because clusterstend to be homogeneous. Cluster designs may employ either simple random or systematic sampling,with or without stratification at any of the stages.

125

Chapter 7 The Logic of Sampling

When clusters are of varying sizes, it is important to vary the procedure by employing probabilityproportionate to size sampling. In this modification, larger clust~s_ar_e_giyen..a_gt~aterch~<:>fbeing selected, but the same number of elements are still selected from each cluster. Sometimes aresearcher may deliberately or inadvertently overrepresent a segment of the population. When this

occurs, w~ghting can be used to correct for the disproportionate sampling.

Ethical considerations in sampling design mean that researchers should point out possible samplingerrors, flaws in the sampling frame, nonresponse error, or other factors that may make the resultsmisleading. When doing qualitative research, the researcher should be careful not to portray a searchfor variation as a study in typicality.

TERMS

1.2.3.

4.5.

6.7.8.

9.10.11.12.13.14.

available subjects sampling (203)cluster sampling (231)confidence interval (219)confidence level (219)disproportionate sampling (236)It (211Y"" \- c~ ,",v\"\iC "'j-V'S <_~\><>~<o\,e emen ~< \<c' co. ,.' "'W~V'c..

EPSEM sample (210)heterogeneity (228)informant (206)nonprobability sampling (203)

(212 <;V~~~nl 0.«'<"'\>""'" o~~parameter )" ...)<~ """""'<>\(. -,,- v~

. d' 'ty (225Y""v~~",\~<>,-,""'-""-peno ICI J<-'c"C~\ ?--",c.c",-

population (211)

probability proportionateto size sampling (235)

15. probability sampling (207)16. purposive sampling (204)17. quota sampling (205)

MATCHING

18.. random selection (212)19 t f (210»0,",," cl('>\-'f\\),",~O.n. «'j-\ '-"~.. represen a Iveness as ~ov ,+- "-(lec \cct ('\7"'20. sampling (200)21. sampling bias (208)22. sampling distribution (213)23. sampling error (217)

• ,--.s~ o\=- eJc"-"'-c\ ....,s %0.-. ,\U"v\:> ~,24. samplmg frame (221) i '0 <--'--')?cACC..\~

25. sampling interval (224~ o.'"\o.,,c<' ",c:~..,)cc:.~\ e \c",<')·(\r nf'_d\

26. sampling ratio (224) .,. c _·dc\,,·d -\~\ ':;c27. sampling unit (212r\C-c-"~ =n~, •28. simple random sampling (224)29. snowball sampling (205) . o( vQ-,io,\q\C,...

.. (216)s"•....M<',., c;'rs({ii71,Q·--.30. statIstIc V"6( t~ ~')\ _poP ['c;.rt<....-.

31. stratification (227)32. study population (211)33. systematic sampling (224)34. weighting (226)

A sampling strategy that involves asking members of a special population for informationneeded to locate others in that population.

----I1.

,/

"1

(\~2.\ /

1(\

3. Another term for probability samples that employ the equal probability of selection meth­od.

The theoretically specified aggregation of survey elements to which results obtained fromthe sample are generalized.

126