29

Performance Improvement, vol. 49, no. 2, February 2010©2010 International Society for Performance Improvement

Published online in Wiley InterScience (www.interscience.wiley.com) • DOI: 10.1002/pfi.20122

DEVELOPING CONTENT-VALID SELECTIONPROCEDURES: WHAT EVERY HPT AND HRD PROFESSIONAL SHOULD KNOW

John C. Cerrito, EdD Gary N. McLean, PhD

To be useful to both human resources development professionals and human performance

technology (HPT) practitioners, the process for developing validated employee selection

instruments must also be user friendly. By following the procedures outlined in this study,

user-friendly pre-employment tests can be developed that are also fair and content valid.

HPT knowledge and support of these procedures will greatly increase the efficiency of the

selection process and ensure that the organizational performers are selected according to

standardized criteria.

PRE-EMPLOYMENT TESTS CAN BE developed thatfocus on knowledge and affective job behaviors andmotor ability skills. Employment tests can also be devel-oped around attributes such as differentiating betweensuccessful and unsuccessful employee job characteristics.The methodological steps for developing validatedemployee pre-employment tests, as outlined in this study,include the following: (a) selecting a job analysis tech-nique (group discussion/brainstorming); (b) establish-ing a diverse committee of job incumbents selected by a human resource professional from the organization;(c) gathering a list of key successful and unsuccessfulcharacteristics from sessions with a committee of jobincumbents; (d) validating the list of job characteristicsbased on the judgments of the incumbents and calculat-ing a content validity index (CVI; Lawshe, 1975) statisticfor each characteristic; (e) developing potential multiplechoice and true/false test items from the validated list ofjob characteristics; (f) validating the potential test itemsthrough the committee and validating each item; and (g) recommending that the organization use the validatedpre-employment test items to hire future employees. Byfollowing the procedures outlined in this study, pre-employment tests can be developed that are fair and con-tent valid and that will greatly increase the efficiency ofthe selection process.

DEVELOPING CONTENT VALIDSELECTION PROCEDURESEmployees are at the center of organizations and areindeed the engines that power successful companies. Thelate Curt Carlson, former CEO of Carlson Companiesand a successful entrepreneur, believed that employeeswere his greatest resource and the reason for his compa-nies’ success (paraphrased from the inscription on abronze statue of Curt Carlson, Carlson CompanyHeadquarters, Minneapolis, Minnesota).

To improve overall organizational productivity andcompetitiveness, employees who will be successful withintheir specific organizations must be chosen. To choosesuccessful employees, organizations must utilize content-valid selection procedures that both optimize the em-ployment process and meet standards for fairness. Thedevelopment of content-valid procedures to assist inselecting the best-qualified employees is legally defensible.When practitioners can easily apply such procedures, theywill make a significant contribution to their organizations.

The cost of replacing, selecting, and training newemployees is high and it is time-consuming for organiza-tions, underscoring the importance of making a goodchoice initially. One study found that the average cost ofrecruiting and training new telephone customer service

30 www.ispi.org • DOI: 10.1002/pfi • FEBRUARY 2010

representatives was between $5,000 and $18,000 per newhire (Anton, 1998). According to O’Connor-Cahill(2002), with the cost of replacing employees increasinglybecoming a concern for organizations, better selectioncriteria need to be utilized. Susan Price, managing editorof HR.BLR.com, is quoted by Campell (2002) as saying“the cost of replacing an employee is usually estimated tobe one third of his or her annual salary.”

New hires who do not fit well in the organization arelikely to become less motivated, less productive, and lesscommitted to the organization. They are more likely toexperience job dissatisfaction and more likely to leave theorganization.

The task of selecting successful employees requires thematching of organizational employment needs with the skills, knowledge, and attitudes of prospective jobapplicants. Successful employee selection is a critical taskthat presents problems and requires attention. Matchingorganizational needs with the qualifications of appli-cants benefits both the employees and the organizationthey work for through increased employee work pro-ductivity and job satisfaction. When developing a pre-employment test, however, the organization must be ableto demonstrate that the test has business relevance(Delikat & Kathawala, 1998).

The development and use of a valid pre-employmenttest, along with other screening tools, such as evaluat-ing resumes and interviews, are effective in selecting em-ployees who will be successful within an organization(Altmann, 1999; Gill, 1994). Pre-employment tests con-sider the knowledge, skills, and attitudes necessary to per-form a specific job (Altmann, 1999; Pickard, 1996).

Rentsch and Hutchison (1999) estimated that morethan 3,000 published tests are available for screening jobapplicants. Whichever pre-employment test an orga-nization uses, the organization must be able to demon-strate that the test reflects the knowledge, skills, and

attitudes required to perform the job within the contextof the organization. Some sources report that the employ-ment test that measures cognitive ability is the best singlepredictor of job performance (Murphy, 2002). However,Terpstra, Mohamed, and Kethley (1999) report that cog-nitive ability tests have been overrepresented in courtcases that challenge selection tests. Much attention hasbeen given to adverse impact, perceived or real, with theuse of cognitive ability tests in the selection process(Reeve, 2002).

The development of content-valid selection instru-ments is of vital concern to all organizations wishing toconduct business in a responsible manner. Employeeturnover due to poor selection tools can result in substan-tial financial expense to the organization. Developing themost effective selection tools possible will result inreduced employee turnover, as well as providing higherproductivity more quickly. As selection tools used byorganizations must comply with guidelines for validityand fairness, selection tools must demonstrate good util-ity, and to do this, they must also be fair, valid, and userfriendly. In developing a content-valid model, two pri-mary considerations had to be taken into account: legaland validity issues.

LEGAL CONSIDERATIONSThe development of a content-valid model with utility forhuman resource development professionals requiresworking within the legal environment of the selectionprocess. To ensure fairness, much of the employee selec-tion process is prescribed by federal legislation andSupreme Court decisions. A stated purpose of these lawsand decisions is to monitor the effect of hiring practicesthat may adversely impact members of protected classesas identified by Title 7 of the 1964 Civil Rights Act Docu-ment Number PL (pp. 88–352).

Adverse impact in the selection process is presumedwhen the pass rate of applicants from a protected class isless than 80% of the pass rate of applicants from thegroup with the highest selection rate. This is referred to asthe “four-fifths” rule. Supreme Court decisions, includingGriggs v. Duke Power Co. (1971) and Albemarle v. Moody(1975), have further stressed the need for developingselection instruments that are directly job-related.

Legislation, key employment law–related decisions, andsocial pressures have led to the establishment of UniformGuidelines on Employee Selection Procedures (FederalRegister, 1978). According to the Uniform Guidelines, allemployment selection devices and instruments, includinginterviews, application forms, pencil-and-paper tests, andphysical exams, are considered to be tests.

By following the proceduresoutlined in this study,pre-employment tests can be developed that are fair andcontent valid and that willgreatly increase the efficiencyof the selection process.

Performance Improvement • Volume 49 • Number 2 • DOI: 10.1002/pfi 31

VALIDITY CONSIDERATIONSThe Uniform Guidelines specifically state that all em-ployment tests must demonstrate job relatedness. Toensure job relatedness, the design and implementation ofselection instruments require validation (Frazee, 1996;Thompson & Thompson, 1982).

There are three key forms of test validity: criterion(predictive/concurrent), construct, and content (seeSociety for Industrial and Organizational Psychology,http://www.siop.org/journal/siopjournal.aspx). Withinthe context of pre-employment testing, test validity usu-ally is defined as the extent to which the criteria used asthe basis for selection are reflective of the key knowledge,skills, and attitudes required to perform a certain job. Asa result, pre-employment tests must focus on job related-ness and meet standards for criterion, construct, or con-tent validation (Sauls, 1995).

With respect to human resource or performanceimprovement applications, criterion validity refers towhether performance scores on one criterion predict theperformance scores on a second criterion. For example,scores on a pre-employment test may or may not pre-dict scores on job performance feedback (job perfor-mance evaluation) or performance in a training program.There are two types of criterion validity: predictive andconcurrent. Predictive criterion validity is established by comparing the results of a pre-employment test withsome other measure of job success, such as trainingresults or on-the-job performance. Concurrent validity isestablished through the same process, except that the testsubjects are current employees (Society for Industrial andOrganizational Psychology, http://www.siop.org/journal/siopjournal.aspx).

Construct validity refers to the measurement of behav-ioral characteristics, qualities, or physiological traits. Todemonstrate construct validity, one must show that a spe-cific trait is critical to performing a particular job, andthen develop or choose some instrumentation to measurethat specific behavioral quality (Mussio & Smith, 1973).

The statistical adequacy of content validity studies isdetermined through the use of Lawshe’s (1975) formula.The application of Lawshe’s formula yields a CVI toestablish statistical significance. For demonstrating thejob relatedness of a selection instrument, content validitystudies are by far the most commonly used (Ebel, 1977;Guion, 1977, 1978). Also, from an applied perspective,content-valid selection instruments have excellent utility(Boudreau, 1992) and, as such, greatly aid in selectingemployees who are the best qualified for particular jobs.

For a pre-employment screening device to be effective,the employment test must be developed around keyknowledge, skills, and attitudes (KSAs) required to per-

form the job. Again, the selection instrument must also becongruent with legal considerations. The selection testshould also differentiate between those potential candi-dates who will become successful employees and thosewho will not be effective on the job. For any of this tomanifest itself, content validity must be established. Theclassic definition of validity (Carmines & Zeller, 1979) isthe extent to which any measuring instrument measureswhat it is intended to measure. Criterion-related validityis useful (Nunnally & Bernstein, 1994) when its pur-pose is to use an instrument to estimate some importantform of behavior that is external to the measuring instru-ment itself, the latter being referred to as the criterion.Criterion-related validity is used to test both concurrentvalidity (the degree of correspondence between the testand the criterion at the time of the testing) and predictivevalidity (the degree of correspondence between the testand some criterion in the future).

VALIDITY GENERALIZATIONThe court decision in the case of Watson v. Fort WorthBank and Trust (1988) allowed for the use of generalizingvalidity among families of occupations in different envi-ronments (Rudner, 1992; Sackett, 1991, 2003).

In essence and in applied terms, the concept of validitygeneralization provides for generalizing a validity studyfor one occupation in a family of jobs to several occupa-tions of jobs within the same job family in other remotelocations (Sackett, 2003). Typically, the pre-employmenttest is developed for a family of factory jobs and not for aspecific factory job occupation. As a result, the pre-employment test can be developed based on general workattitudes rather than specific job skills.

Prior to the 1988 Supreme Court decision of Watson v.Fort Worth Bank and Trust, all pre-employment tools usedby employers were supposed to be developed as a result oflocally conducted validity studies (Rudner, 1992). Localvalidity refers to the concept that validity can be deter-mined only in a specific occupational, situational environ-ment. This concept required that new validation studies beconducted whenever a pre-employment test is applied to anew work environment (Schmidt & Hunter, 1977).

In the Watson decision, the courts ruled that employ-ers can cite other studies conducted in other environ-ments in similar occupations and populations todemonstrate the validity of pre-employment selectiontools (Rudner, 1992). As an example, an employer maychoose to utilize a pre-employment test that was devel-oped in another environmental setting for a family of fac-tory jobs that are similar to the factory jobs in theemployer’s own environment, assuming such a test exists.

32 www.ispi.org • DOI: 10.1002/pfi • FEBRUARY 2010

The concept of validity generalization provides that pre-employment tests are developed around general abilitiesrather than occupation-specific job functions (Biller,1993). Indeed, Perlman, Schmidt, and Hunter (1980) statedthat general cognitive ability can be a good predictor ofperformance for occupations within a family of jobs. Linnand Dunbar (1986) stated, with respect to requiring sepa-rate validity studies for separate job occupations, that theredoes not exist sufficient evidence to demonstrate thatrequirement. This observation is consistent with Salgado’s(1997) study that researched the variability in validity coef-ficients in situational validity studies and in validity gener-alization examples.

It is still the burden of the employer to demonstratethat a business need exists for the use of a specific selec-tion instrument. It is the employer’s responsibility todemonstrate that the selection test is job-related and, as aresult, is relevant to occupations within the employer’sorganization.

Using general intelligence tests broadly as selectioninstruments, without demonstrating job relatedness, cancreate legal exposure for the organization. As an example,courts have rejected the use of general intelligence testsfor use in selecting firefighters (Rudner, 1992). Rudnerfurther reported that public employers in almost 40 stateswere using pre-employment selection tests developed orbased on generalized validity as opposed to situationalvalidity studies. In a more recent study based on organi-zations in the European Community, general ability testswere found to be good predictors of job performanceacross a variety of occupations (Salgado, Anderson,Moscoso, Bertua, & De Fruyt, 2003).

Sackett (1991) explored criteria for clustering militaryoccupations and for generalizing relationships within an occupation to a larger universe of specialties. Thewidespread use of the General Aptitude Test Battery inmaking selection decisions raised issues of fairness withrespect to the impact on protected classes (Sackett, 2003).The National Academy of Sciences Committee examinedthe issue of fairness and validity generalization and,although accepting the general thesis, criticized severalspecific applications of validity generalization (Murphy,2002; Sackett, 2003).

After the use of selection tools can be generalized, inreality, the employer still has the obligation to demon-strate that the selection instruments used by the orga-nization have relevance to the job. With respect toemployment practices, all organizations should be inter-ested in promoting fairness. Using valid, job relevantemployment instruments will increase organizationalperformance and reduce possible legal exposure.

The goal for human resource development (HRD)professionals is to use procedures to develop pre-employment tests that meet standards for fairness and arecontent valid. This can be accomplished by implementingthe following procedures as indicated in Table 1.

PROCEDURE STEP 1: JOB ANALYSISConducting a job analysis requires gathering data fromthose who performed the work. The primary resource forgathering key job criteria, such as work attitudes, knowl-edge, and skills required to do the job (job characteris-tics), must be the job incumbent.

TABLE 1 PROCEDURAL COMPONENTS ADDRESSING EEOC/FAIRNESS AND CONTENT VALIDITY

CONSIDERATION PROCEDURAL COMPONENT

EEOC fairness Committee of job incumbents is formed. Committee reflects the diversity of the organization’s workforce.

Job analysis technique is selected.

Committee develops list of key employee job attitudes and characteristics (relevant job criteria).

Committee then ranks the importance of each key characteristic.

Consensus is established.

Based on the committee’s ranking, a content validity index (Lawshe, 1975) is calculated on each employee characteristic.

Content-valid job characteristics are established as a result of CVI score.

Pre-employment test items are developed from the list of employee characteristics with statistically relevant CVI scores.

Committee ranks job relevancy of developed test items.

Pre-employment test developed from ranked test items.

Content validityjob characteristics

Content validity pre-employmenttest items

Performance Improvement • Volume 49 • Number 2 • DOI: 10.1002/pfi 33

The diversity––gender, race, age, job station, andlongevity––of the job incumbents who develop the keyjob criteria must reflect the diversity of the organization’sworkforce. This group will ultimately provide key inputduring the job analysis process and reflects the richness ofdiversity within the company. Employees serving on theincumbent committee are referred to as subject matterexperts (SMEs). Although the majority of employees pro-viding job information must be job incumbents, com-pany supervisors/managers should also provide input.

Gathering data from employees requires the selectionof a job analysis technique. EEOC guidelines do notrequire the use of a particular job analysis technique.Useful techniques in conducting job analysis includeinterviews, observations, questionnaires, and group dis-cussions (Schmidt, Gooding, Noe, & Kirsch, 1984).

One acceptable job analysis method is a group discus-sion that employs a nominal group technique (Delbecq,van de Ven, & Gustafson, 1975). This technique requiresthat each group member provide key job information,which is accomplished by proceeding in a clockwisedirection around the table and calling on each member ofthe group for input. This pattern is repeated several times,and a brainstorming session follows.

PROCEDURE STEP 2: ADDRESS VALIDITYCONSIDERATIONSAddressing validity considerations is critical to the devel-opment of successful pre-employment instruments.Establishing content validity is a key step in developingselection procedures.

As indicated in Procedure Step 1, company SMEsshould be asked to identify key characteristics. After thegathering of key employee job characteristics, each indi-vidual SME on the committee of job incumbents is thenasked to judge the importance of each key characteristic.

Before the SMEs make such judgments, the committeemembers should attend orientation and question-and-answer sessions to become familiar with the validationprocess and the rating scales they would use to makejudgments. This orientation session will be useful in help-ing the SMEs to understand and apply a rating scale whenmaking judgments on each job characteristic. One sug-gested rating scale that can be used is as follows: 3 =Extremely Important, 2 = Very Important, 1 = SomewhatImportant, 0 = Not Important.

The above-mentioned rating scale can be used todetermine a content validity statistic for each employeecharacteristic judged by the SMEs by calculating a contentvalidity index using Lawshe’s (1975) formula: CVI (con-tent validity index) = (Ne – N/2)/N/2), in which Ne = the

number of experts indicating that an item is essential (3)or helpful (2), and N = the total number of raters.

Specifically, the application of Lawshe’s (1975) CVIformula requires that the SME committee evaluate the listof characteristics previously developed by the commit-tee. Each committee member should then be asked tojudge each characteristic with respect to how importantthat characteristic is to performing as a successful com-pany employee.

PROCEDURE STEP 3: DEVELOPPOTENTIAL TEST ITEMSThe development of potential pre-employment test itemsis based on the employee characteristics that are judgedby the SMEs to be important, according to CVI statistics,as indicated in Procedure Step 2. A test-constructionblueprint needs to be developed for the purpose of docu-menting the relationship between the characteristicsjudged to be statistically significant by the SMEs and thedevelopment of potential pre-employment test items.

PROCEDURE STEP 4: VALIDATEPOTENTIAL TEST ITEMSThis step requires that the committee of SMEs validate allpotential pre-employment test items. Specifically, eachSME of the incumbent committee will be asked to judgethe extent to which each potential pre-employment testitem represents a key employee characteristic (as identi-fied in Procedure Step 2). The SMEs should also be askedto judge the extent to which each test item would distin-guish potentially successful company employees frompotentially unsuccessful company employees.

Based on the judgments of the SME committee members, Lawshe’s (1975) formula can now be used tocalculate a content validity statistic (CVI score) for eachpotential pre-employment test item. Test items with thehighest CVI scores then become the foundation ofthe organization’s pre-employment test or questionnaire.

SUMMARYBoth HRD and human performance technology (HPT)professionals need to know the legal parameters associ-ated with employee selection and performance testing.The development of pre-employment tests requires care-ful attention to employment law issues including EEOCguidelines and validity requirements. To ensure contentvalidity, criteria used as the basis for making employmentdecisions must be job related. The selection instrumentitself must reflect key job criteria. Key job criteria are a

34 www.ispi.org • DOI: 10.1002/pfi • FEBRUARY 2010

function of the KSAs (characteristics) required in perform-ing the job. For both the human resource and performanceimprovement specialist, a job analysis is typically used togather information about the key KSA requirements of ajob. Conducting a job analysis requires the input of thosemost familiar with the performance of job duties. Theinclusion of job incumbents is a vital part of the process ofdeveloping selection criteria. The job incumbent’s role inthe process is to serve as a SME. It is the prerogative of theSME to both identity and validate job criteria.

Developing selection tools requires following certainspecific procedures as indicated in Table 1. The purposeof following this specific procedure is to ensure compli-ance of U.S. Equal Employment Opportunities Commis-sion guidelines (1978) and to satisfy content validityrequirements.

Having employee involvement is a key activity and isalso a good example of reaffirming the organization’scommitment to partnering with employees to improvethe culture of work.

Effectively matching the needs of the organization withabilities and career interests of employees can increaseemployee job satisfaction. For the performance improve-ment professional, helping organizations make better hir-ing decisions can also result in increased organizationalproductivity and performance. Finally, better hiring deci-sions can reduce typical recruitment, selection, andturnover costs, which ultimately help increase employees’job performance (Pucel, Cerrito, & Noe, 1989).

References

Albemarle v. Moody. (1975). Retrieved August 6, 2007, fromhttp://laws.findlaw./us/000/97–282html.

Altmann, R. (1999). Why use pre-employment assessments?Management Journal. Retrieved April 7, 2007, fromhttp://www.assessments.ncs.com.

Anton, J. (1998). Benchmarking study. Datamonitor. RetrievedMay 1, 2003, from http://datamonitor.com.

Biller, E.F. (1993). Employment testing of persons with specificlearning disabilities. The Journal of Rehabilitation, 59(1), 16–18.

Boudreau, J. (1992). Utility analysis for decisions in humanresource management. Handbook of Industrial andOrganizational Psychology, 2(4), 621–745.

Campbell, B. (2002). The high cost of turnover: Why holdingon to your employees can improve your bottom line. The FreeLibrary. Retrieved May 1, 2009, from http://www.thefreelibrary.com/The high cost of turnover: why holding on to youremployees can. . .-a094672514.

Carmines, E.G., & Zeller, R.A. (1979). Reliability and validityassessment. Beverly Hills, CA: Sage.

Civil Rights Act, Title 7, Document No. PL88-352 (1964).

Delbecq, A.L., van de Ven, A.H., & Gustafson, D.H. (1975).Group techniques for program planning: A guide to nominal andDelphi processes. Glenview, IL: Scott Foresman.

Delikat, M., & Kathawala, R. (1998). Personality and aptitudetests: Insurance against hiring mistakes or invitation to litiga-tion? Managing Office Technology, 43(2), 16–20.

Ebel, R.I. (1977). Proposed solutions to two problems of testconstruction. Journal of Educational Measurement, 19, 267–268.

Federal Register. (1978). Retrieved fromhttp://www.archives.gov/federal_register/publication.

Frazee, V. (1996). Do your job-applicant tests make the grade?Personnel Journal, 75(6), S16–17.

Gill, B. (1994). Pretesting reveals more. American Printer,212(5), 92.

Griggs v. Duke Power Co. 401 US424. (1971). RetrievedSeptember 29, 2003, from http://laws.findlaw.com/us.401/424.html.

Guion, R.M. (1977). Content validity––The source of my dis-content. Applied Psychological Measurement, 1(1), 1–10.

Guion, R.M. (1978). Content validity in moderation. PersonnelPsychology, 31(2), 205–214.

Lawshe, C.H. (1975). A quantitative approach to content valid-ity. Personnel Psychology, 28(4), 563–575.

Linn, R.L., & Dunbar, S.B. (1986). Validity generalization andpredictive bias. In R.A. Berk (Ed.), Performance assessment(pp. 228–247). Baltimore, MD: The Johns Hopkins UniversityPress.

Murphy, K. (2002). Can conflicting perspectives on the role ofG in personnel selection be resolved? Human Performance,15(1/2), 173–186.

Mussio, S.J., & Smith, M.K. (1973). Content validity: A proce-dure manual. Minneapolis, MN: Civil Service Commission.

Nunnally, J.C., & Bernstein, I.H. (1994). Psychometic theory.New York: McGraw-Hill.

O’Connor-Cahill, M.E. (2002). A study of how the congruencebetween person and culture related to employee perceptions oforganizational fit, job satisfaction, organizational commit-ment, and intent to leave. Dissertation Abstracts International:Section B: The Sciences & Engineering, 62, 4769. Retrieved May21, 2004, from http://cal.csa.com.

Performance Improvement • Volume 49 • Number 2 • DOI: 10.1002/pfi 35

Perlman, K., Schmidt, F.L., & Hunter, J.E (1980). Validity gen-eralization results for tests used to predict job proficiency andtraining success in clerical occupations. Journal of AppliedPsychology, 65(4), 376–406.

Pickard, J. (1996). The wrong turns to avoid with tests. PeopleManagement, 2(16), 92–96.

Pucel, D.J., Cerrito, J.C., & Noe, R. (1989). Integrating selec-tion, training and performance. Performance ImprovementQuarterly, 2(4), 21–29.

Reeve, C.L. (2002). Asking the right questions about g. HumanPerformance, 15(1/2), 47–74.

Rentsch, J., & Hutchison, S. (1999). Testing the test. HR Focus,76, 13.

Rudner, L.M. (1992). Pre-employment testing and employeeproductivity. Public Personnel Management, 21(2), 133.

Sackett, P.R. (1991). Exploring strategies for clustering militaryoccupations. In A.K. Wigdor & B.F. Green (Eds.), Performanceassessment for the workplace, II (pp. 305–332). Washington,DC: National Academy Press.

Sackett, P.R. (2003). The structure of counterproductive workbehaviors: Dimensionality and relationship with facets of jobperformance. International Journal of Selection and Assessment,10(1), 5–11.

Salgado, J.F. (1997). The five factor model of personality andjob performance in the EC. Journal of Applied Psychology,82(1), 30–43.

Salgado, J.F., Anderson, N., Moscoso, S., Bertua, C., & De Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European community meta-analysis. Personnel Psychology, 56(3), 573–605.

Sauls, J. (1995). Establishing the validity of employment stan-dards. The FBI Law Enforcement Bulletin, 64(1), 27–33.

Schmidt, N., Gooding, R., Noe, R., & Kirsch, M. (1984). Meta-analysis of validity studies published between 1964 and 1982and the investigation of study characteristics. PersonnelPsychology, 37(3), 407–422.

Schmidt, N., & Hunter, J. (1977). Measurement error in psy-chology research: Lessons from 26 research scenarios.Psychological Methods, 1, 199–223.

Terpstra, D.E., Mohamed, A.A., & Kethley, B. (1999). Legalchanges associated with different selection devices.International Journal of Selection and Assessment, 7(1),26–34.

Thompson, D.E., & Thompson, T.A. (1982). A court standardfor job analysis in test validation. Personnel Psychology, 35(4),865–874.

U.S. Equal Employment Opportunities Commission, CivilService Commission, Department of Justice. (1978). Uniformguidelines on employee selection procedures. Federal Register,43, 38290–38315.

Watson v. Fort Worth Bank & Trust, 487 U.S. 977. (1988).Retrieved September 24, 2003, from http://findlaw.us/000.

JOHN C. CERRITO, EdD, is a professor in the department of business administration at AugsburgCollege, Minneapolis, Minnesota. Former chair of the business department, he provided leadershipfor the undergraduate program and guided the development of the master’s in business administra-tion degree program. He has been the recipient of four distinguished teaching awards, including aBoard of Regents Teaching Award. He has lectured on business management and human resourcesdevelopment topics at the State University of Management, Moscow, Russia; Catholic University,Lublin, Poland; and Oxford University, Oxford, England. He researches and writes in the area ofhuman resource development. He may be reached at cerritoj@augsburg.edu.

GARY N. MCLEAN, PhD, is a senior professor and executive director of international human resourcesprograms at Texas A&M University, and former professor and coordinator of human resources devel-opment at the University of Minnesota, St. Paul. He has served as president of the Academy of HumanResource Development and the International Management Development Association. He is installed in the Scholar Hall of Fame of the Association of Human Resource Development and in theInternational Adult and Continuing Hall of Fame. As an organizational development practitioner inMcLean Global Consulting, Inc., he works extensively worldwide. His research interests are broadand diverse, focusing primarily on organization development and international HRD. He may bereached at gmclean@tamu.edu.