Extreme Groups DesignsKristopher J. PreacherVanderbilt University

It is common for psychological researchers toexamine the relationship between two variablesX and Y by first selecting cases on the basisof extreme X scores (e.g., X scores situatedin the outer tertiles or quartiles of the scoredistribution) and then using X to predict Y.For example, a researcher may administer adepression scale to a participant pool, ran-domly select participants from the lower andupper 25% of the distribution of depressionscores, and then use depression as a predictorof alcohol consumption for the sampled cases.This sampling procedure has been termed theextreme groups approach (EGA), and it resultsin an extreme groups design. The extremegroups design can be seen as a special case ofoversampling, in which the outer portions of adistribution are sampled to the complete exclu-sion of the central portion. The extreme groupsdesign also represents a special case of restric-tion of range, in which the range of sampled Xscores is restricted to the extremes of the dis-tribution of X, with predictable consequencesfor subsequent statistical analyses. In addition,the extreme groups design is also a specialcase of observational or quasi-experimentaldesign because participants are measuredon X, not randomly assigned to treatmentconditions.

Several modifications of EGA exist. Forexample, the extreme groups need not be equalin size, nor must they span the same rangeof scores in the lower and upper extremes ofthe score distribution. Most often, the extremegroups are chosen on the basis of sample quan-tiles, but also can be selected on the basis ofconventional cutoff points or features of thescore distribution. Often, the extreme groups,once selected, are assigned codes for "low"

The Encyclopedia of Clinical Psychology, First Edition. Edited by Robin L. Cautin and Scott O. Lilienfeld.© 2015 John Wiley & Sons, Inc. Published 2015 by John Wiley & Sons, Inc.DOI: 10.1002/9781118625392.wbecp190

(0) vs. "high" (1) prior to analysis, creatingtwo groups, each of which is assumed to behomogeneous on X. EGA does not describestudies that limit attention to one extremeor the other (such as studies of academicallygifted children). Nor does it describe designsin which data are collected across the full rangeof X and then discarded from the middle of theX distribution.

Past Investigations of ExtremeGroups Designs

EGA has a long history in the literature onresearch design and applied statistics. Earlyuses in the 1930s through 1950s were motivatedby attempts to enhance statistical power ofsubsequent tests (Kelley, 1939), simplify thetask of fitting a line to data (Bartlett, 1949;Gibson & Jowett, 1957; Wald, 1940), andreduce the cost of data collection (Peters, 1941;Peters & Van Voorhis, 1940). These reasonslikely number among the most popular rea-sons for employing EGA in more moderntimes as well.

Consequences of Using ExtremeGroups Designs

Cost-EfficiencyCost—in terms of time, money, or labor—isoften a limiting factor in data collection.Assuming a fixed sample size and a desiredlevel of power for detecting a relationship,EGA can improve cost-efficiency by reducingthe sample size required to achieve the desiredpower. Improved cost-efficiency is one of thefew clear benefits of EGA, but it is beneficialonly if the researcher is interested in hypoth-esis testing and is willing to assume that thesame model that applies to the extremes ofthe X distribution also applies to the omittedmiddle.

2 EXTREME GROUPS DESIGNS

Statistical PowerPower for the test of the correlation of X andY is generally enhanced after using EGA onX. All else being equal, power increases withthe extremity of selection on X. For example,power for the test of rXY will be greater if Ncases are selected from the outer quartiles thanif they are selected from the outer tertiles. Thisdifference in power is more pronounced forsmaller N. Many studies spanning the 1920s to1970s (e.g., Feldt, 1961) showed that the powerfor a t test comparing the mean of Y acrossgroups created on the basis of extreme X scoresis maximized when the proportion of casesdrawn from each extreme is approximately27%, nearly a quartile split (this came to beknown as the "twenty-seven percent rule").However, higher power can be achieved bynot grouping after EGA, instead preservingthe individual differences within each extremeof X and using correlation or regression togauge the relationship between X and Y (Alf &Abrahams, 1975).

Effect SizeEGA has no biasing effect on the covarianceor regression slope relating X and Y. However,EGA usually increases measures of standard-ized effect size (e.g., r, R2) relative to usingdata from the full range of X (Cortina andDeShon 1998). This inflated effect size leadsto higher power for tests of linear association,but also gives a misleading picture of the sizeor practical significance of the effect. Thus,standardized effect sizes derived from extremegroups designs should not be generalized tothe population or used in meta-analyses. Thisinflationary effect is most pronounced forcorrelations near .5 and is attenuated as r nears0 and ±1 (Preacher, Rucker, MacCallum, andNicewander, 2005).

ReliabilityEGA is often thought to improve reliabilitythrough selection of the X scores perceived tobe the most reliable (the extremes). However,the opposite is generally true; it can be shownthrough application of item response theory

principles that scores in the interior of the Xdistribution tend to be those that are the mostreliable (Preacher et al. 2005). In addition,scores in the extremes of the X distribution arethose that are most vulnerable to regressiontoward the mean. That is, the extreme X scoresare those that are the most liable to changeby a noteworthy amount upon repeated mea-surement, and thus have the lowest test–retestreliability.

Model MisspecificationEGA removes from consideration the middleportion of the distribution of X. Thus, the(typically linear) relationship that is observedto hold between X and Y when using extremegroups must be assumed to also hold for casesin the omitted portion of the X distribution.Because this segment of the population hasbeen omitted from consideration, it is notpossible to explore a possibly nonlinear rela-tionship. EGA typically increases the power ofsubsequent tests of linear relationships betweenX and Y, but at the expense of rendering thedetection of nonlinear effects improbable.

Group AssignmentSometimes the extreme scores of a continuousX distribution are recoded into a binary (0,1)variable to represent groups that are low andhigh on X, which then are used in subsequentanalyses (e.g., a t test for group means onY). There are multiple arguments against thispractice. It assumes that (1) two underlyingcategories really exist and that observed indi-vidual differences reflect only measurementerror rather than meaningful individual differ-ences; (2) group assignment is accurate; (3) theproportions of cases in each group are equal inthe population; and (4) group constituency isstable over samples, contexts, and occasions.Furthermore, true dichotomies are arguablyrare in psychological research. The implica-tion is that, if EGA is to be used, individualdifferences within each extreme should be pre-served, and regression-based methods shouldbe used rather than a t test. If the existence ofunobserved groups is plausible, more suitable

EXTREME GROUPS DESIGNS 3

methods exist for determining whether scoreson X reflect a discrete vs. continuous variable.

Interaction EffectsSometimes EGA is used to create subgroups foruse in subgroup analysis, a common tool forinvestigating interaction effects. In subgroupanalysis, groups are created on the basis ofextreme scores on a moderator variable Z, andthe relationship between X and Y is assessedin each group. If the relationship of X and Y isfound to differ between Z groups, an interac-tion is said to exist. Whereas this procedure isintuitive and simple to implement, the creationof subgroups on Z has many of the limitationsmentioned above for group creation. Method-ologists have suggested more appropriate andpowerful methods for detecting interactionsthat do not require the creation of arbitrarygroups, such as moderated multiple regression(MMR) with product terms. If EGA is usedin conjunction with MMR—that is, if extremecases are sampled on both X and Z, and theseextreme scores are treated as continuous pre-dictors in a regression analysis predicting Yfrom X, Z, and the product XZ—the powerfor detecting interaction effects can be greatlyenhanced (McClelland and Judd, 1993).

Potential Applications of ExtremeGroups Designs

On balance, the usefulness of EGA as amethodological tool is somewhat limited. Sev-eral costs have been described above. However,there are valid reasons researchers may want toemploy extreme groups designs in some situ-ations. As indicated, costs associated with datacollection may prohibit the examination of datafrom the full range of X. In such cases, EGAcan be used to increase the power for detectinga linear association given a fixed sample size.EGA can be useful for enhancing power notonly for detecting bivariate relationships, butalso interactions. It should be borne in mind,however, that using EGA to enhance power fora fixed N carries with it several dangers andassumptions. Typically, only linear associations

may be explored after EGA, reliability is almostalways reduced relative to using data from thefull range of X, and the researcher is obligatedto avoid interpreting standardized effect size.Due to its power-enhancement properties,EGA may find its best use in pilot studies,where the goal is to detect promising trends insmall samples that may warrant further inves-tigation. When EGA is used, its use should bethoughtfully considered and explicitly justified.Researchers who are unaware of the limitationsof extreme groups designs are in danger ofoverstating results.

A method closely related to EGA is posthoc subgrouping. In contrast to EGA, whichinvolves collecting N data points from arestricted range of X scores, post hoc sub-grouping involves deleting from a data setthose cases having X scores in the interior ofthe X distribution, and sometimes dichotomiz-ing the remaining cases into low and highgroups. This practice shares the disadvantagesassociated with EGA and lacks its advantages.Unlike EGA, subgrouping does not improvecost-efficiency (the data have already beencollected). Second, because data are discardedin subgrouping, the power to detect linearrelationships relative to analyses using the fullrange of X scores is diminished.

SEE ALSO: Construct Validity; CorrelationalDesigns; Distinguishing Categories from Continua;Effect Size; Item Response Theory; Quasi-Experi-ments; Psychopathology Research, Selection ofSubjects in; Statistical Power

ReferencesAlf, Jr, E. F., & Abrahams, N. M. (1975). The use of

extreme groups in assessing relationships.Psychometrika, 40, 563–572.

Bartlett, M. S. (1949). Fitting a straight line whenboth variables are subject to error. Biometrics, 5,207–212.

Cortina, J. M., & DeShon, R. P. (1988). Determiningrelative importance of predictors with theobservational design. Journal of AppliedPsychology, 83, 798–804.

Feldt, L. S. (1961). The use of extreme groups to testfor the presence of a relationship. Psychometrika,26, 307–316.

4 EXTREME GROUPS DESIGNS

Gibson, W. M., & Jowett, G. H. (1957).’Three-group’ regression analysis: Pt. I. Simpleregression analysis. Journal of the RoyalStatistical Society, Series C (Applied Statistics), 6,114–122.

Kelley, T. L. (1939). The selection of upper andlower groups for the validation of test items.Journal of Educational Psychology, 30, 17–24.

McClelland, G. H., & Judd, C. M. (1993). Statisticaldifficulties of detecting interactions andmoderator effects. Psychological Bulletin, 114,376–390.

Peters, C. C. (1941). A technique for correlatingmeasurable traits with freely observed socialbehaviors. Psychometrika, 6, 209–219.

Peters, C. C., & Van Voorhis, W. R. (1940).Statistical procedures and their mathematicalbases. New York: McGraw-Hill.

Preacher, K. J., Rucker, D. D., MacCallum, R. C., &Nicewander, W. A. (2005). Use of the extremegroups approach: A critical reexamination andnew recommendations. Psychological Methods,10, 178–192.

Wald, A. (1940). The fitting of straight lines if bothvariables are subject to error. Annals ofMathematical Statistics, 11, 284.

Further ReadingAbrahams, N. M., & Alf, Jr, E. F. (1978). Relative

costs and statistical power in the extreme groupsapproach. Psychometrika, 43, 11–17.

Borich, G. D., & Godbout, R. C. (1974). Extremegroups designs and the calculation of statisticalpower. Educational and PsychologicalMeasurement, 34, 663–675.