SelectionSelectionMANA 3320MANA 3320

Dr. Jeanne MichalskiDr. Jeanne Michalskimichalski@uta.edu

The Goal of Selection: Maximize “Hits”The Goal of Selection: Maximize “Hits”

MISSMISS InaccurateInaccuratepredictionprediction

(Person would have (Person would have succeeded on the job)succeeded on the job)

HITHIT AccurateAccuratepredictionprediction

(Person succeeds on (Person succeeds on the job)the job)

HITHIT AccurateAccuratepredictionprediction

(Person would not (Person would not have succeeded on the have succeeded on the

job)job)

MISSMISS Inaccurate predictionInaccurate prediction

(Person fails on the (Person fails on the job)job)

Predicted Success

Job

Perfo

rman

ce

Low

Low

High

High

“Earns a Bonus”

“Is a Bonehea

d”

The Selection ProcessThe Selection Process Obtaining Reliable and Valid InformationObtaining Reliable and Valid Information

ReliabilityReliability The degree to which interviews, tests, and other The degree to which interviews, tests, and other

selection procedures yield comparable data over time selection procedures yield comparable data over time and alternative measures.and alternative measures.

ValidityValidity Degree to which a test or selection procedure Degree to which a test or selection procedure

measures a person’s attributes.measures a person’s attributes.

Reliability: Basic ConceptsReliability: Basic Concepts Observed score = true score + errorObserved score = true score + error

Error is anything that impacts test scores Error is anything that impacts test scores that is that is notnot the characteristic being the characteristic being measuredmeasured

Reliability measures error Reliability measures error

Lower the error the better the measureLower the error the better the measure

EEOC Uniform GuidelinesEEOC Uniform GuidelinesReliability – consistency of the measureReliability – consistency of the measure

If the same person takes the test again will If the same person takes the test again will he/she earn the same score?he/she earn the same score?

Potential contaminations:Potential contaminations: Test takers physical or mental stateTest takers physical or mental state Environmental factorsEnvironmental factors Test formsTest forms Multiple ratersMultiple raters

Reliability as Stability over Reliability as Stability over TimeTime

HIGH RELIABILITYHIGH RELIABILITY TESTTEST RETEST RETEST APPLICANTAPPLICANT SCORESCORE SCORESCORESmithSmith 9090 9393PerezPerez 6565 6262RileyRiley 110110 105105ChanChan 8080 7878

VERY LOW RELIABILITYVERY LOW RELIABILITY TESTTEST RETESTRETESTAPPLICANTAPPLICANT SCORESCORE SCORE SCORESmithSmith 9090 7272PerezPerez 6565 8888RileyRiley 110110 6767ChanChan 8080 111111

Reliability as ConsistencyReliability as Consistency(Interrater Reliability)(Interrater Reliability)

HIGH RELIABILITYHIGH RELIABILITY

APPLICANTAPPLICANT Rater #1Rater #1 Rater #2Rater #2 Rater #3Rater #3SmithSmith 99 88 88PerezPerez 55 66 55RileyRiley 44 55 55ChanChan 88 88 88

VERY LOW RELIABILITYVERY LOW RELIABILITY

APPLICANTAPPLICANT Rater #1Rater #1 Rater #2Rater #2 Rater #3Rater #3SmithSmith 99 55 66PerezPerez 55 99 44RileyRiley 44 22 77ChanChan 88 44 22

CorrelationCorrelation How strongly are two variables related?How strongly are two variables related?

Correlation coefficient (r)Correlation coefficient (r) Ranges from -1.00 to 1.00Ranges from -1.00 to 1.00

Illustrated using scatter plotsIllustrated using scatter plots Used to test consistency and accuracy of Used to test consistency and accuracy of

measuremeasure

Correlation ScatterplotsCorrelation Scatterplots

Figure 5.3

Reliability vs. ValidityReliability vs. Validity

Approaches to ValidationApproaches to Validation Content validityContent validity

The extent to which a selection instrument, such as The extent to which a selection instrument, such as a test, adequately samples the knowledge and skills a test, adequately samples the knowledge and skills needed to perform a particular job.needed to perform a particular job.

Example: typing tests, driver’s license examinations, work Example: typing tests, driver’s license examinations, work samplesample

Construct validityConstruct validity The extent to which a selection tool measures a The extent to which a selection tool measures a

theoretical construct or trait.theoretical construct or trait. Are difficult to validateAre difficult to validate

Example: creative arts tests, honesty testsExample: creative arts tests, honesty tests

Approaches to ValidationApproaches to Validation Criterion-related ValidityCriterion-related Validity

The extent to which a selection tool predicts, or The extent to which a selection tool predicts, or significantly correlates with, important significantly correlates with, important elements of work behavior.elements of work behavior.

A high score indicates high job performance A high score indicates high job performance potential; a low score is predictive of low job potential; a low score is predictive of low job performance.performance.

Two types of Criterion-related validityTwo types of Criterion-related validity Concurrent Validity Concurrent Validity Predictive ValidityPredictive Validity

Approaches to ValidationApproaches to Validation Concurrent ValidityConcurrent Validity

The extent to which test scores (or other predictor The extent to which test scores (or other predictor information) match criterion data obtained at about the information) match criterion data obtained at about the same time from current employees.same time from current employees.

High or low test scores for employees match their respective job High or low test scores for employees match their respective job performance.performance.

Predictive ValidityPredictive Validity The extent to which applicants’ test scores match criterion The extent to which applicants’ test scores match criterion

data obtained from those applicants/ employees after they data obtained from those applicants/ employees after they have been on the job for some indefinite period.have been on the job for some indefinite period.

A high or low test score at hiring predicts high or low job A high or low test score at hiring predicts high or low job performance at a point in time after hiring.performance at a point in time after hiring.

Reliability vs. ValidityReliability vs. Validity Validity CoefficientsValidity Coefficients

Reject below .11Reject below .11 Very useful above .21Very useful above .21 Rarely exceed .40Rarely exceed .40

Reliability CoefficientsReliability Coefficients Reject below .70Reject below .70 Very useful above .90Very useful above .90 Rarely approaches 1.00Rarely approaches 1.00

Why the difference?Why the difference?

Principles of AssessmentPrinciples of Assessment Don’t rely on a single method.Don’t rely on a single method. Use only fair and unbiased instruments. Use only fair and unbiased instruments. Use only reliable instruments.Use only reliable instruments. Use only valid instruments for a specific purpose.Use only valid instruments for a specific purpose. Use only tools designed for a specific group.Use only tools designed for a specific group. Use instruments with understandable instructions.Use instruments with understandable instructions. Ensure test administration staff are properly trained.Ensure test administration staff are properly trained. Ensure test conditions are suitable for all test takers.Ensure test conditions are suitable for all test takers. Provide reasonable accommodation.Provide reasonable accommodation. Maintain confidentiality of results.Maintain confidentiality of results. Ensure proper interpretation of results.Ensure proper interpretation of results.

Steps in the Selection Steps in the Selection ProcessProcess

Initial screening of application or resumeInitial screening of application or resume Employment testing, if applicableEmployment testing, if applicable Interview (may include multiple visits for Interview (may include multiple visits for

interviews)interviews) HR HR Supervisor/TeamSupervisor/Team OthersOthers

Contingent OfferContingent Offer Background/Reference ChecksBackground/Reference Checks Medical/Drug screenMedical/Drug screen

Application FormsApplication FormsOnly ask info related to job KSA’s

Link to job performance Use thorough job analysis and validation techniques Consider potential adverse impact

Careful collecting personal characteristics Race, National Origin, Gender, Age etc. Law assumes all questions are used in hiring More is not necessarily better

Different applications for different jobs Instructions and Disclaimers Improve validity through weighted applications

(WAB’s)

Online ApplicationsOnline Applications An Internet-based automated posting, An Internet-based automated posting,

application, and tracking process helps application, and tracking process helps firms to more quickly fill positions by:firms to more quickly fill positions by:

Attracting a broader and more diverse applicant Attracting a broader and more diverse applicant poolpool

Collecting and mining resumes with keyword Collecting and mining resumes with keyword searches to identify qualified candidatessearches to identify qualified candidates

Conducting screening tests onlineConducting screening tests online Reducing recruiting costs significantlyReducing recruiting costs significantly

ResumesResumes Applicant controls the informationApplicant controls the information

Many examples of fraud or omissionMany examples of fraud or omission Up to 50% contain some inaccuracyUp to 50% contain some inaccuracy

Jobs and education should be verifiedJobs and education should be verified One question honesty testOne question honesty test

Requirements for education and Requirements for education and experiences should be job-related. experiences should be job-related.

Biodata InventoriesBiodata Inventories Generally reliable and validGenerally reliable and valid Series of questions on a wide variety of Series of questions on a wide variety of

subjectssubjects Background (e.g. hobbies, jobs, and education)Background (e.g. hobbies, jobs, and education) Situational questionsSituational questions Personality and ValuesPersonality and Values

Compared to a profile generated from Compared to a profile generated from successful employees or database for successful employees or database for occupations.occupations.

Examples of Biographical Examples of Biographical QuestionsQuestions

How many jobs have you held in the last five years? Were you ever class president? While growing up, did you collect coins? About how many fiction books have you read in the past

year? How many hours a week do you spend studying? At what age did you leave home? How large was the town/city in which you lived as a child? Did you ever build a model airplane that flew? Were sports a big part of your childhood? Do you play any musical instruments?

Employment TestsEmployment Tests Employment TestEmployment Test

An objective and standardized measure of a An objective and standardized measure of a sample of behavior that is used to gauge a sample of behavior that is used to gauge a person’s knowledge, skills, abilities, and other person’s knowledge, skills, abilities, and other characteristics (KSAOs) in relation to other characteristics (KSAOs) in relation to other individuals.individuals.

Pre-employment testing hasPre-employment testing hasthe potential for lawsuits. the potential for lawsuits.

Classification of Classification of Employment TestsEmployment Tests

Cognitive Ability TestsCognitive Ability Tests Aptitude testsAptitude tests

Measures of a person’s capacity to learn or acquire Measures of a person’s capacity to learn or acquire skills.skills.

Achievement testsAchievement tests Measures of what a person knows or can do right now.Measures of what a person knows or can do right now.

Personality and Interest InventoriesPersonality and Interest Inventories ““Big Five” personality factors:Big Five” personality factors:

Extroversion, agreeableness, conscientiousness, Extroversion, agreeableness, conscientiousness, neuroticism, openness to experience.neuroticism, openness to experience.

Classification of Classification of Employment Tests (cont’d)Employment Tests (cont’d)

Physical Ability TestsPhysical Ability Tests Must be related to the essential functions of Must be related to the essential functions of

job.job. Job Knowledge TestsJob Knowledge Tests

An achievement test that measures a person’s An achievement test that measures a person’s level of understanding about a particular job.level of understanding about a particular job.

Work Sample Tests Work Sample Tests Require the applicant to perform tasks that are Require the applicant to perform tasks that are

actually a part of the work required on the job.actually a part of the work required on the job.

Effectiveness of Selection MethodsEffectiveness of Selection MethodsA survey of 201 HR executives rated selection methods on the effectiveness producing the best employees.

Work samples 3.68Structured interviews 3.42Assessment centers 3.42Specific aptitude tests 3.08Personality tests 2.93General cognitive ability tests 2.89Biographical information blanks 2.84Unstructured interview < 1.00

Common Background Common Background ChecksChecks

Common Background Common Background ChecksChecks

EducationEducation EmploymentEmployment ReferencesReferences Criminal HistoryCriminal History Credit CheckCredit Check Drug ScreenDrug Screen Medical ScreenMedical Screen

Drug testingDrug testing Urinalysis vs. RIAH (Hair testing)Urinalysis vs. RIAH (Hair testing) ADA prohibits pre-employment screening for alcoholADA prohibits pre-employment screening for alcohol Drug Free Workplace Act of 1988Drug Free Workplace Act of 1988

Credit checksCredit checks Depends on the job – e.g. cash handling, children.Depends on the job – e.g. cash handling, children. Signed consent to background or credit checks.Signed consent to background or credit checks.

Fair Credit Reporting Act Fair Credit Reporting Act Family Educational Rights Privacy Act Family Educational Rights Privacy Act Privacy Act of 1974Privacy Act of 1974

Employee Polygraph Protection Act (1988)Employee Polygraph Protection Act (1988)

““Screening-out” DevicesScreening-out” Devices

Background ChecksBackground Checks Negligent hiringNegligent hiring

The failure of an organization to discover, via due diligence, The failure of an organization to discover, via due diligence, that an employee it hired had the propensity to do harm to that an employee it hired had the propensity to do harm to othersothers

Sources of InformationSources of Information Social Security verificationSocial Security verification Past employmentPast employment Educational verificationEducational verification Criminal recordsCriminal records Motor vehicle recordsMotor vehicle records Credit checkCredit check Military recordsMilitary records

Negligent HiringNegligent Hiring The person causing harm was an employee.The person causing harm was an employee. The person was incompetent (rather than a The person was incompetent (rather than a

competent employee acting negligently).competent employee acting negligently). Employer knew or should have known about the Employer knew or should have known about the

incompetence.incompetence. Injury or harm was a foreseeable consequence of Injury or harm was a foreseeable consequence of

hiring the incompetent employee.hiring the incompetent employee. Hiring of the unfit employee was proximate cause Hiring of the unfit employee was proximate cause

of the injury or harm.of the injury or harm.

Reference Checks Reference Checks Telephone, mail, and e-mail checksTelephone, mail, and e-mail checks

Specific job-related informationSpecific job-related information Letters of referenceLetters of reference Online computerized databasesOnline computerized databases Privacy Act of 1974Privacy Act of 1974

Requires signed requests for reference letters and Requires signed requests for reference letters and signed consent to background checks.signed consent to background checks.

Applies to both educational and private employers.Applies to both educational and private employers.

Employee Polygraph Employee Polygraph Protection Act (1988)Protection Act (1988)

Use of “lie Use of “lie detectors” is largely detectors” is largely prohibited.prohibited.

Act requires Act requires qualified examiners.qualified examiners.

Act requires Act requires disclosure of disclosure of information where information where used.used.

Encouraged Encouraged employers’ use of employers’ use of paper and pencil paper and pencil integrity and integrity and honesty tests.honesty tests.

Background Investigations Background Investigations Organizations using credit reports must:Organizations using credit reports must:

1.1. Check state laws to see if credit reports can legally be used.Check state laws to see if credit reports can legally be used.2.2. Advise and receive written consent from applicants if a Advise and receive written consent from applicants if a

report will be requested.report will be requested.3.3. Provide a written certification to the consumer reporting Provide a written certification to the consumer reporting

agency as to the purpose of the report.agency as to the purpose of the report.4.4. Provide applicants a copy of the consumer report as well as Provide applicants a copy of the consumer report as well as

a summary of their rights under the CCRRA.a summary of their rights under the CCRRA.5.5. Must provide an adverse-action notice a person if that Must provide an adverse-action notice a person if that

person is not hired and contact information related to the person is not hired and contact information related to the reporting agency. reporting agency.

Criminal Background Criminal Background ChecksChecks

Employers can obtain criminal background information by: Employers can obtain criminal background information by: (1) Asking applicants for certain information during the interview (1) Asking applicants for certain information during the interview

process or on a job applicationprocess or on a job application(2) Obtaining criminal record information from a state agency (2) Obtaining criminal record information from a state agency

(with the consent of the applicants)(with the consent of the applicants)(3) by hiring an investigator or security firm(3) by hiring an investigator or security firm

Before the information is gathered, employers should have Before the information is gathered, employers should have specific guidelines for how they will use the information.specific guidelines for how they will use the information.

Decision CriteriaDecision CriteriaThe hiring official will determine on a case-by-case basis whether The hiring official will determine on a case-by-case basis whether

the individual is qualified based on factors such as:the individual is qualified based on factors such as: Specific duties of the position; Specific duties of the position; Number of offenses; Number of offenses; Nature of each offense; Nature of each offense; Length of time intervening between the offense and the Length of time intervening between the offense and the

employment decision; employment decision; Employment history; Employment history; Efforts at rehabilitation; and Efforts at rehabilitation; and Accuracy of the information that the individual provided on the Accuracy of the information that the individual provided on the

employment applicationemployment application

Background Investigations Background Investigations (cont.)(cont.)

Medical ExaminationsMedical Examinations Given last as they can be costly.Given last as they can be costly. Ensure that the health of an applicant is Ensure that the health of an applicant is

adequate to meet the job requirements.adequate to meet the job requirements. Provides a baseline for subsequent Provides a baseline for subsequent

examinationsexaminations ADA requires all exams be job-related and ADA requires all exams be job-related and

conducted after an employment offer is made.conducted after an employment offer is made. Testing for illegal drugs is allowed.Testing for illegal drugs is allowed.

Drug TestingDrug Testing Drug-Free Workplace Act of 1988Drug-Free Workplace Act of 1988

Testing for illegal drugs is required of applicants Testing for illegal drugs is required of applicants and employees of federal contractors.and employees of federal contractors.

Questions about the efficacy of testingQuestions about the efficacy of testing Why spend large sums on testing when…Why spend large sums on testing when…

testing for drugs doesn’t appear to make the testing for drugs doesn’t appear to make the workplace safer or improve employee performance?workplace safer or improve employee performance?

few applicants actually test positive and alcohol abuse few applicants actually test positive and alcohol abuse creates more problems in the workplace?creates more problems in the workplace?

Most Common Types of Background Checks

Use of Pre-Employment Background Screening by Fortune 1000 Companies*Use of Pre-Employment Background Screening by Fortune 1000 Companies*

*Tools that 212 security representatives at Fortune 1000 companies said their companies use consistently.

Source: Top Security Threats and Management Issues Facing Corporate America, Pinkerton Consulting and Investigations, Inc.

Principles of AssessmentPrinciples of Assessment Don’t rely on a single method.Don’t rely on a single method. Use only fair and unbiased instruments. Use only fair and unbiased instruments. Use only reliable instruments.Use only reliable instruments. Use only valid instruments for a specific purpose.Use only valid instruments for a specific purpose. Use only tools designed for a specific group.Use only tools designed for a specific group. Use instruments with understandable instructions.Use instruments with understandable instructions. Ensure test administration staff are properly trained.Ensure test administration staff are properly trained. Ensure test conditions are suitable for all test takers.Ensure test conditions are suitable for all test takers. Provide reasonable accommodation.Provide reasonable accommodation. Maintain confidentiality of results.Maintain confidentiality of results. Ensure proper interpretation of results.Ensure proper interpretation of results.