Testing in Testing in Industrial & Industrial &

Business Business SettingsSettings

How are people hired for a How are people hired for a particular job or position?particular job or position?

Employment interviewEmployment interview Can be influenced by extraneous factor Can be influenced by extraneous factor

such as age, disability status, gender, such as age, disability status, gender, racerace

Use of testsUse of tests When used in addition to interview & When used in addition to interview &

other sources of information, can other sources of information, can improve chances of successful selectionimprove chances of successful selection

Base Rates & Hit RatesBase Rates & Hit Rates

tests often used to make dichotomous decisions: Pass/fail Accept/reject Hire/don’t hire

E.g., use of an 1100 score on the GRE to decide on whom to reject or whom to process further

This value is a cut-off score or a cutting score

The problem with cutting The problem with cutting scoresscores

Some people who have scores above the cutting score will do poorly

Some people who have scores below the cutting score would have done well

Hits & MissesHits & Misses

HitHit MissMiss

MissMiss HitHit

Decision on basis of cutting scoreAccep

tReject

Actual Outcome

SuccessFailure

Hit RateHit Rate

Percentage of cases in which a test Percentage of cases in which a test accurately predicts success or accurately predicts success or failurefailure

Hits & MissesHits & Misses

2020 4040

1010 3030

Decision on basis of cutting scoreAccep

tReject

Actual Outcome

SuccessFailure

What is the hit rate?

Base RateBase Rate

The selection or pass rate obtained The selection or pass rate obtained withoutwithout using the test using the test

E.g., using only undergraduate E.g., using only undergraduate grades, letters of reference, etc. (no grades, letters of reference, etc. (no GRE), 70% of individuals selected GRE), 70% of individuals selected are successful in completing their are successful in completing their degreedegree

If hit rate using the test is only 60%, If hit rate using the test is only 60%, test is of no valuetest is of no value

False negatives & false False negatives & false positivespositives

There are two types of “misses”:There are two types of “misses”: False negativeFalse negative

Individuals not selected (because of a score below the Individuals not selected (because of a score below the cutting score) who would have been successful in job cutting score) who would have been successful in job or program (e.g., GRE has high false negative rate)or program (e.g., GRE has high false negative rate)

Concluding on the basis of a test that a tumor is Concluding on the basis of a test that a tumor is benignbenign

Concluding on basis of a test that someone is not Concluding on basis of a test that someone is not suicidal, when he/she actually issuicidal, when he/she actually is

False positiveFalse positive Individuals selected for job or program who failIndividuals selected for job or program who fail Concluding on the basis of a test that someone has Concluding on the basis of a test that someone has

AIDS, when they do not have the diseaseAIDS, when they do not have the disease

HitHit False False negativenegative

FalseFalse

PositivePositiveHitHit

Decision on basis of cutting score

Hire as pilot

Reject

Actual Outcome

Success

Failure

What is more dangerous, false +ve or false –ve?

HitHit False False negativenegative

FalseFalse

PositivePositiveHitHit

Decision on basis of cutting scoreDiagnose

Aids

Diagnose

No Aids

Actual DiseaseStatus

Has AIDS

Doesn’t

Have AIDS

What is more dangerous, false +ve or false –ve?

Costs of False NegativesCosts of False Negatives& False Positives& False Positives

For certain decisions, false negatives For certain decisions, false negatives will have a high costwill have a high cost E.g., diagnosing a person as not having E.g., diagnosing a person as not having

AIDS when she really does have it; that AIDS when she really does have it; that a person is not suicidal when he really isa person is not suicidal when he really is

For other decisions, false positives For other decisions, false positives will have a high costwill have a high cost E.g. hiring someone as a commercial E.g. hiring someone as a commercial

pilot when he is likely to crash the planepilot when he is likely to crash the plane

Test Performance

Actual

Outcome

Cutting Score

Success

Failure

False -ve

False +ve

False negative, positive & criterion validity

Changing Rate of False Changing Rate of False Negatives & False Positives Negatives & False Positives

by adjusting the cutting by adjusting the cutting scorescore

To reduce the number of false To reduce the number of false negatives,negatives, lower lower the cutting scorethe cutting score

To reduce the number of false To reduce the number of false positives, positives, raise raise the cutting score the cutting score

AA

88BB

22

CC

1515DD

7575

Decision on basis of cutting scoreDiagnose

Aids

Diagnose

No Aids

Actual DiseaseStatus

Has AIDS

Doesn’t

Have AIDS

Detection rate (sensitivity) = A/(A+B) = 8/10 Accuracy Rate = (A+D)/(A+B+C+D) = 83/100 (83%)

Taylor Russell TablesTaylor Russell Tables

helps evaluate validity of test in relation to amount it contributes beyond base rates

Requires Definition of success -- patient lived, success on

job, school Determination of base rate Definition of selection ratio -- % applicants

selected Validity coefficient

table gives likelihood that person selected on basis of test score will be successful

Taylor-Russell Table for a Base Rate of .60Taylor-Russell Table for a Base Rate of .60

Class ExerciseClass Exercise Suppose that you are director of graduate studies in psychology

at Wilfrid Laurier University. You can choose 10 students for the graduate program. A total of 50 students apply to the program, and normally you would make your decision on the basis of undergraduate grades, letters of recommendation, work and volunteer experiences, and the like. In the past, about 60% of the students whom you admit complete their degrees. Up to this point, your program has not required applicants to submit GRE scores, but you are considering adding a GRE requirement for new applications, even though it correlates only about .25 with success in graduate school.

What is the base rate of success in this instance?

WWhat is your success rate adding the GRE?

How many people will you reject that would probably have been successful in the program? (i.e., how many false negatives will there be?)

How many people will you accept who will likely not complete the program? (how many false positives will there be?)

77 2323

33 1717

Decision on basis of cutting scoreAccep

tReject

Actual Outcome

SuccessFailure

Base rate - .60

Selection ratio - .20

Validity coefficient - .25

30

20

10 40 50

Tests for Use in IndustryTests for Use in Industry

Bennett Mechanical Comprehension Bennett Mechanical Comprehension TestTest Mechanical abilityMechanical ability

Minnesota Clerical TestMinnesota Clerical Test Clerical skillsClerical skills

Computer Programmer Aptitude Computer Programmer Aptitude BatteryBattery Computer programming skillsComputer programming skills

Integrity TestsIntegrity Tests

Also known as “honesty tests”Also known as “honesty tests” Paper-and-pencil inventories used for Paper-and-pencil inventories used for

personnel selection to identify potentially personnel selection to identify potentially dishonest or “counterproductive” dishonest or “counterproductive” employeesemployees

American businesses lose from $15- to American businesses lose from $15- to $25-billion per year due to employee theft$25-billion per year due to employee theft

30% of all business failures attributed to 30% of all business failures attributed to employee theftemployee theft

Personnel Selection Personnel Selection Inventory (PSI)Inventory (PSI)

Includes scales that measure:Includes scales that measure: HonestyHonesty Drug avoidanceDrug avoidance Tenure or job commitmentTenure or job commitment SafetySafety Work valuesWork values Supervision attitudesSupervision attitudes Validity Validity

CandidnessCandidness Accuracy Accuracy

Sample ItemsSample Items

Responses are recorded on 7-point scale ranging from “far more than average” (1) to

“far less than average” (7)

Validity of PSIValidity of PSI In 1982 & 1983 a chain of home improvement centers screened In 1982 & 1983 a chain of home improvement centers screened

more than 4,000 applicants with PSI to help reduce on-the-job more than 4,000 applicants with PSI to help reduce on-the-job theft and drug abuse.theft and drug abuse.

PSI not used from 1979 to 1981PSI not used from 1979 to 1981 ““shrinkage” losses reduced from $7.5 million to $5.25 millionshrinkage” losses reduced from $7.5 million to $5.25 million Number of employees terminated for using drugs at work reduced Number of employees terminated for using drugs at work reduced

from 16 in 1981 to 1 in 1983 from 16 in 1981 to 1 in 1983

Measuring Characteristics Measuring Characteristics of the Work Settingof the Work Setting

Based on the premise that work Based on the premise that work environments will influence the environments will influence the productivity and satisfaction of individuals productivity and satisfaction of individuals who work in those environmentswho work in those environments

Rudolph Moos developed scales to measure Rudolph Moos developed scales to measure several different kinds of environments:several different kinds of environments: Work environmentsWork environments ClassroomsClassrooms Correctional institutionsCorrectional institutions

Work Environment ScaleWork Environment Scale

10 subscales, organized into 3 10 subscales, organized into 3 dimensions:dimensions: Relationship Relationship Personal growth Personal growth System maintenance & system changeSystem maintenance & system change

Relationship DimensionRelationship Dimension InvolvementInvolvement (extent to which employees are (extent to which employees are

concerned about & committed to their jobs)concerned about & committed to their jobs) ““people seem to take pride in the organization”people seem to take pride in the organization”

Peer cohesionPeer cohesion (extent to which employees (extent to which employees are friendly & supportive of one another)are friendly & supportive of one another) ““There’s not much group spirit”There’s not much group spirit”

Supervisor supportSupervisor support (extent to which (extent to which management is supportive of employees)management is supportive of employees) ““supervisors tend to talk down to employees”supervisors tend to talk down to employees”

Personal Growth Personal Growth DimensionDimension

AutonomyAutonomy (extent to which employees are (extent to which employees are encouraged to be self-sufficient)encouraged to be self-sufficient) ““people can use their own initiative to do things”people can use their own initiative to do things”

Task orientationTask orientation (degree of emphasis on (degree of emphasis on good planning, efficiency & getting the job good planning, efficiency & getting the job done)done) ““this is a highly efficient, work-oriented place”this is a highly efficient, work-oriented place”

Work pressureWork pressure (degree to which the press of (degree to which the press of work & time urgency dominate the job work & time urgency dominate the job milieu)milieu) ““there is constant pressure to keep working”there is constant pressure to keep working”

System Maintenance & System Maintenance & System Change DimensionSystem Change Dimension

ClarityClarity (extent to which employees know what to (extent to which employees know what to expect)expect) ““things are sometimes pretty disorganized”things are sometimes pretty disorganized”

ControlControl (extent to which management uses rules & (extent to which management uses rules & pressures to keep employees under control)pressures to keep employees under control) ““Supervisors keep a rather close watch on employees”Supervisors keep a rather close watch on employees”

Innovation Innovation (degree of emphasis on variety, change, (degree of emphasis on variety, change, new approaches)new approaches) ““New & different ideas are always being tried out”New & different ideas are always being tried out”

Physical comfortPhysical comfort (extent to which physical (extent to which physical surroundings contribute to pleasant work surroundings contribute to pleasant work environment)environment) ““work space is awfully crowded”work space is awfully crowded”

Sample WES ProfilesSample WES Profiles