Criteria

Definition: Standards used to make evaluative judgments about objects, people, or events.

Criteria serve as the basis by which judgments can be made.

An example - Waiter/Waitress Many different criteria can be (and are) applied:

General speed. Friendliness Being Observant (e.g., noticing empty glasses). Appearance Dependability

What Criteria do you use when buying a car?

Conceptual v. Actual Criteria

Conceptual criteria – the theoretical standard that researchers seek to understand through their research.

Actual criteria - serves as a measure of the conceptual criteria.

Example CC: Administrative assistant performance. This global

construct would probably include hundreds of smaller criteria (e.g., being courteous to people on the phone, taking accurate and complete messages, typing correspondence quickly and correctly…).

AC: Performance ratings made by his or her boss.

Criterion Deficiency, Relevance, and Contamination

Criterion Deficiency - The degree to which the actual criteria fail to overlap the conceptual criteria. In other words, it is how deficient the actual criteria are in representing the conceptual ones.

Criterion Relevance - The degree to which the actual criteria and the conceptual criteria coincide.

Criterion Contamination - The part of the actual criteria that is unrelated to the conceptual criteria. It is the extent to which the actual criteria measure something other than the conceptual criteria.

Graphic Example

Graphic Example

Some Causes of Deficiency, Relevance, and Contamination

Deficiency Activities performed by the administrative

assistant that the boss does not directly witness. For example, the boss doesn’t hear the assistant talking on the phone (usually) and must therefore base this rating on limited information, such as the fact that he has never received a complaint about the assistant.

Activities overlooked by the boss when making ratings.

Some Causes of Deficiency, Relevance, and Contamination

Contamination Bias - the extent to which the actual criteria systematically

or consistently measure something other than the conceptual criteria. For example, the administrative assistant is very nice to the boss, causing a generally positive impression which may be unrelated to actual job performance.

Error - The extent to which the actual criteria are not related to anything at all.

Relevance The boss is present for many of the assistant’s activities,

and the assistant’s performance presumably has a direct impact on the boss’s ability to perform his own job.

Uses of Criteria

Training success Selection Promotions Raises Employee feedback In effect, every major decision made is based on

some criteria. Due to this wide use, the quality of criteria is very

important.

Proximal v. Distal Criteria

Proximal Criteria - Short-term effects. Distal Criteria - Long-term effects. Important to take time into consideration.

Selection Validation Training Evaluation Other areas.

Quality of criteria is important because poor criteria lead to poor judgments.

Job Analysis

Definition - A process undertaken to determine the responsibilities of a job and the associated knowledge, skill, and ability requirements (KSAs).

Why conduct a job analysis? Criterion Development (e.g., performance appraisal) Reduction of role conflict and ambiguity. Personnel Selection Training Design Wage and salary determinations.

Job Analysis

What is a job? Job Family - A group of generally similar jobs. Job - group of positions which are similar enough to

warrant one analysis, may cover one or many people. Position - A specific person’s set of tasks (Every person

in an organization has a different position). Task - work activity

Example Job Family - Autoworker Job - Assembly Line Worker Position - Welder Task - Welds together body panels using blow torch.

Job Analysis

Sources of Job Information Existing written material (e.g., training

manuals, job descriptions, etc…). Observe individuals doing the job. Interview Subject Matter Experts (SMEs)

Incumbents - employees currently in the job (preferably those who have a lot of experience).

SupervisorsTrainersPrevious incumbents (preferably those who have

only recently moved out of the job).

Job Analysis

How do you choose? Availability - often your choice is limited. Who you

interview usually depends on who’s available at the time.

Motivation - Who will be motivated to provide complete information?

Objectivity - Are there reasons why a particular SME would be biased?

Often try to have multiple sources of information to validate the information collected.

Task Oriented Procedures

A task oriented procedure seeks to understand a job by examining the tasks performed, usually in terms of what is accomplished. Tasks are the basic unit of analysis. Task statements are used to describe job.

Functional Job Analysis is Task Oriented Determine WHAT worker does and HOW task is

performed. Describes the content of jobs in terms of data,

people, and things (see Figure 3-3 on page 63 of text).

Task Statements

Task statements should tell what the work is (verb), how it is done, to whom or what and why. They should begin with a strong verb.

Examples Sorts correspondence forms and reports in

order to facilitate filing them alphabetically. Informs next shift supervisor of departmental

status through written or verbal reports.

Work Oriented Procedures

Work oriented procedures seek to understand a job y examining the human attributes needed to perform it successfully. Focuses on KSAOs

Linkage Analysis – unites the two basic types of job analysis information: task and work-oriented. Examines relationships between KSAOs and tasks performed.

KSAs

Knowledge: what the person needs to know in order to perform the job. Example: Knowledge of the restaurant’s menu.

Skill: What the person needs to know how to do in order to perform the job. Example: Skill in being able to handle customer

complaints in a polite and courteous fashion.

Abilities: what the person needs to be able to do in order to perform the job. Example: Ability to carry a full serving tray over one’s

head for several minutes at a time.

KSA Examples

Knowledge of vehicle paints, primers, and sealant and their application.

Knowledge of local building codes. Knowledge of database software. Skill in reading a blueprint. Skill in operating a cash register. Ability to remain objective and professional in

stressful or personally unpleasant interpersonal situations.

Ability to work quickly.

Job Analysis Methods

Interview SMEs (most common). Questionnaire (Position Analysis Questionnaire). Direct observation - better for blue collar than

white collar jobs. Work diaries. Some combination of the above techniques.

Interview Tips

Explain what you are doing and why. Start with a basic idea so you can ask some

leading questions. It is difficult (impossible?) for people to remember and state every tasks and KSA. By asking the right questions, you will get fewer omissions.

Interview in a quiet place where there won’t be as many interruptions.

Don’t be afraid of silence - your interviewee needs time to think.

Taxonomic Sources of Information

Position Analysis Questionnaire (PAQ) 195 statements used to describe the human attributes

needed to perform a job. The statements are organized into six major categories: information input, mental processes, work output, relationships with other persons, job context, and other requirements.

Fleishman’s 52 abilities Examples: oral expression, reaction time, selective

attention

Dictionary of Occupational Titles (DOT) Occupational Information Network (O*NET).

An on-line computer based source of information about jobs.

Job Performance Criteria

Objective Criteria Examples: Production units, sales, tenure,

turnover, absenteeism, accidents, theft. Advantages

Easy measurement.Impartial.

DisadvantagesOften only part of the story.Meaning may not be straight-forward (Example:

turnover).Better for some jobs than others.

Job Performance Criteria

Subjective Criteria Usually ratings or rankings. May be general or specific (note: ratings

usually more accurate when focused on specific aspects of the job).

General: overall effectiveness.Specific: quality of work.

Most common - supervisory ratings 360 degree feedback

Ratings provided by supervisor, peers, subordinates, self, clients, etc…

Provides unique perspectives.

Job Performance Criteria

Dynamic Criteria Most criteria are dynamic, they change over

time (e.g., accidents, absence, productivity, etc…). However, we often treat criteria as static entities and may therefore not get an accurate picture.

Other issues Intercorrelations among job criteria. Complexity

More criteria needed for more complex jobs.

Developing Criteria

Bartender Performance What kinds of objective criteria could you use?

What kinds of subjective criteria could you use?

What are the problems with these?

Critical Incidents Interview Technique

Ask subject matter expert to think of specific incidents illustrating extremely poor or extremely effective performance.

Determine performance dimensions reflected by incidents.

Can be very useful for specific purposes. Toledo Police BARS development.

Critical Incident Questions

What were the circumstances leading to the incident? Example: It was a Friday night and the restaurant was

extremely busy. Every table was full. One of the servers was sick and called out of work.

What did the employee do that made you think she or he was a good or poor performer? Example: the employee picked up the sick server’s tables

and waited on them in addition to his regular tables. The service he provided to all tables was excellent.

What were the consequences of the operator’s behavior in the incident? Example: As a result, the restaurant was able to keep all the

tables full that night and all the customers were satisfied.