a question of evidence - user's guide - telephone doctor€¦ · web viewuse the...

A Question of EvidenceThe Behavior-Based Interview

By

Hugh Murray FCIPD

Subject consultant

Colin Chase FCIPD

2 © 2006 Telephone Doctor, Inc. – Not to be distributed outside North America

User’s Guide to:A Question of Evidence The Behavior-based Interview

Produced and published by:Newmarket LearningA Division of Telephone Doctor, Inc.30 Hollenberg Ct.St. Louis, MO 63044 Under exclusive license issued by Scott Bradbury, LTD, UK

Telephone: 314.291.1012Fax: 314.291.3710Web: www.newmarketlearning.com

First Published 2006Reprint 1 2 3 4 5 6 7 8 9

http://www.scottbradbury.co.uk/

Contents Contents.............................................................................................................................................3Copyright............................................................................................................................................4Introduction........................................................................................................................................5How to use this program....................................................................................................................7Half-day training session....................................................................................................................9Role-play…………………………………………………………………………………………………… 12A short guide to behavior-based interviewing..................................................................................13Analysis of interviews.......................................................................................................................19Behavioral indicators........................................................................................................................24James’ notes on Shawna.................................................................................................................25Candidate's resume'………………………………………………………………………………………..26Post-course test...............................................................................................................................27Post-course test answers.................................................................................................................28

© 2006 Telephone Doctor, Inc. – Not to be distributed outside North America 3

Copyright

The moral issue

In all likelihood, our company is a smaller organization than your own. We take a substantial risk every time we produce new training material and we spend a great deal of time and money on developing material that helps your organization get better at what it does. If someone uses our material without paying us, they are stealing our livelihood. If allowed to proceed unchecked, copyright theft will lead to unemployment in our industry and the loss of valuable training materials to society as a whole. Copying is dishonest. It is theft. It is unfair. And it usually involves larger organizations stealing from smaller ones.

The legal issue

This guide, the DVD and the PowerPoint® presentation are all copyrighted material. The copyright belongs to Telephone Doctor, Inc.

You are welcome to copy parts of this guide and to edit the PowerPoint® presentation provided you acknowledge the source as being Telephone Doctor, Inc.

You may never copy the DVD, not even as a backup. We will replace the DVD if it becomes unplayable within five years of purchase, even if this is the result of carelessness or accident. All you need do is send the unplayable DVD back to us and we will send you a replacement without charge.

The purchaser of the DVD has a license to use it for internal training purposes only. It may not be used in public events, or lent to other organizations. It may be sold provided that you notify Scott Bradbury of the sale and provided that the subsequent purchaser agrees to the same license terms as the original purchaser.


Introduction

Behavior and competency

Behavior and competency are two terms that you need to understand in order to understand behavior-based interviewing. We have simplified them in a way that we think makes them easier to use in a training context. We know that this is a gross simplification of the ideas but it works for us. We hope it works for you.

Behavior

A behavior is an instance of what someone does and how he or she does it. For example: gives clear directions in a crisis while remaining calm.

Competency

A competency is a cluster of related behaviors that drives effectiveness in a particular area of work. Examples are: leadership, teamwork and communication. Each one would be defined by a list of behaviors at different levels.

Behavior-based interviewing

Behavior–based interviewing involves being clear about what competencies you want a candidate to have in the job and then looking for evidence that he or she has them.

Many interviewers ask about a candidate’s experience. Experience alone is not a good predictor of how a person will perform in a new job. The fact that someone has done something tells you very little about how well they have done it. Also, success in one situation in one environment does not guarantee success in a different situation in a different environment. If this were true, there would be some people who were successful at every job and some people who were unsuccessful at every job. In reality, most people make a success of some things that they do and fail at others. That is because success is a combination of the person, the situation and the environment.

You want to know if a candidate will likely succeed in your situation and in your environment. You can do this by being clear about what behaviors – what people actually do and how they do it – work well in your situation and environment and then looking for evidence that a candidate displays those behaviors.

More about competencies

You don’t need to have a competency framework in place to use behavioral questions in your interviews. But it helps. Competency is a psychological term describing an underlying trait or characteristic that leads to effective performance at work. It might be a motivation, a personality trait, a prejudice, a value, a skill, an attitude or knowledge. Or it could be a combination of all of these things. It is the stuff inside you that makes you behave in a particular way.

This is different than competence. Competence is the condition of being capable. A competent brain surgeon is someone who is able to do the job satisfactorily. .


In a groundbreaking study in 1982, Richard Boyatzis (Professor of Organizational Behavior, Case Western University, Cleveland, Ohio), set out to find out what it was that made some managers successful while others struggled. This study is published in his book The Competent Manager, and one of the foundations of the competency movement. He defined successful managers as those whose peers and superiors believed them to be successful and he set out to find out what made them tick.

Boyatzis eventually identified 21 competencies that were present to varying degrees in successful managers. He gave them names such as Efficiency Orientation and Use of Socialized Power. He then analyzed each competency or underlying characteristic in terms of a person’s motive, his or her self-image and skill.

This is heady stuff. Much of it is far too obscure to be of use to today’s practicing manager. However, one powerful idea that has emerged from his work is this. We cannot hope to understand all the things that make us behave the way we do. But we can observe behavior and behavior is remarkably constant. When we know someone well, we can usually predict how he or she will behave with a fair degree of reliability. We may not know why they behave in a certain way, but we know that they do.

Many organizations today have set about studying what competencies make for success. They have identified the factors that they believe make people successful in their particular environment and situation. They have given these competencies names, such as Teamwork, and they have identified behavioral indicators that define them in terms of the way people work, such as works cooperatively and effectively with others in a complex political and social environment. These competencies are usually defined in terms of level, too, so that teamwork for the board of directors requires higher-order behaviors than those required of a production team.

A modern organizational framework consists of a list of competencies – personal factors that lead to success – and a description of the behaviors at different levels that result from those competencies. These are called behavioral indicators. If your organization has them then a large part of the preparation for a behavioral interview is done for you. You already know what behaviors you are seeking. You can concentrate on finding evidence that your candidate does, or does not, display them.


How to use this program

What is in this program?

DVD - A Question of Evidence This manual PowerPoint® presentation

Who is the program for?

Any person who carries out selection interviewing or who is likely to do so.

What does this program do?

Defines and discusses behavior-based interviewing Promotes an understanding of the principles involved Demonstrates how people can ask behavioral questions effectively.

How does this program achieve its objectives?

As a resource which your organization can use in several ways:

For individual self-study As part of an existing training program As a complete half-day training session.

Individual self-study

Individuals can use the DVD and a copy of A short guide to behavior-based interviewing, as an introduction to the subject or a reminder of the important principles.

In an existing training program

The program has been designed as a resource for your existing programs. You can use the DVD as a training video in the conventional sense. Or you can use the different interviews as the basis for exercises and discussions. We have provided you with an analysis of each interview that you can use to prepare your own exercises. And you are welcome to use the PowerPoint® presentation and the component parts of this manual in any way that suits you.

A complete half-day training program

This manual contains trainer’s notes for a complete training session lasting three hours and ten minutes.


Evaluation of learning

We have included a post-course test on page 27 along with an answer key on page 28. This will help you measure whether your participants have understood the key ideas. Role-play on page 12 will help you assess the participants’ skill in using the techniques presented in the program.


Half-day training session

Learning objectives

After this half-day training session, participants will be able to:

1. explain the difference between behavioral, biographical and hypothetical questions and be able to explain the advantages and disadvantages of each.

2. devise an effective opening behavioral question; given a set of behavioral indicators (an effective opening behavioral question is one that causes a candidate to disclose useful information about a critical incident).

3. apply the P.R.O.B.E structure (see page 17 for an explanation of acronym), given a set of behavioral indicators and a candidate, in order to discover evidence that the candidate either does, or does not, display the behaviors required.

Resources

Meeting room DVD player and monitor DVD A Question of Evidence PowerPoint® presentation and projector Flip chart and pens Job Materials

o Copies of James’ notes on Shawna page 25o Copies of candidate resume page 26o oro Copies of your organization’s competency frameworko Copies of anonymous candidate’s resume

Number and background of participants

Maximum of 16. Participants should be responsible for selection interviewing or should be about to undertake that responsibility.

Trainer preparation

Review the session plan Read A short guide to behavior-based interviewing page 13 Review the PowerPoint® presentations and prepare your notes Watch the DVD Take the test

Participant preparation (what you need to do for participants)

Read A short guide to behavior-based interviewing on page 13 before the course. Select a real job vacancy and review the job specification. If there is currently no vacancy,

select a job that is currently filled. Bring the job specification to the session. Review the organization’s competency framework relevant to the chosen job.


Room layout

Seating should allow the participants to work as a group and also in sub-groups of four. It must be possible for everyone to see the DVD, PowerPoint Screen and flip chart.

Timetable

Time Elapsed timeIntroduction 10 minutes 10 minutesDVD 20 minutes 30 minutesBehavioral and other questions 15 minutes 45 minutesExamples of questions 20 minutes 1 hour 05 minutesAnalyzing questions 10 minutes 1 hour 15 minutesBreak 15 minutes 1 hour 30 minutesAdvantages of types of question 15 minutes 1 hour 45 minutesUsing different types of question 15 minutes 2 hoursDevising an opening question 15 minutes 2 hours 15 minutesP.R.O.B.E. for evidence 15 minutes 2 hours 30 minutesPutting it all together 30 minutes 3 hoursSummary and close 10 minutes 3 hours 10 minutes

Introduction

Welcome participants in your usual way, including any introductions and explanations of procedures that might be required.

Explain that the session will cover:

What behavioral questions are Why they are beneficial How to use them.

Show the DVD A Question of Evidence. Use the play complete video option.

Behavioral and other questions

Use the accompanying PowerPoint® presentation to explain what is:

A behavioral question A biographical question A hypothetical question.

Small group exercise – examples of questions

Divide participants into sub-groups of four. Ask the sub-groups to choose a job for which they might recruit and then to devise:

● Three biographical questions Three behavioral questions Three hypothetical questions.

Allow ten to fifteen minutes for this, or until the noise dies down.


Invite members of each sub-group to volunteer their questions. Capture the questions on a flip chart. Ask members of the whole group to give their opinions on the questions. Participants agree that they are examples of the relevant type of question? If not, why not? Could the questions be improved and, if so, how?

Large group exercise – analyzing questions

Play the interview Randall from the DVD menu.

Pause the DVD after each question in the interview and ask the members of the group to tell you what type of question it is. Refer to the section Analysis of Interviews on page 19 to help you prepare for this.

Small group exercise – advantages of each type of question

Divide participants into sub-groups of four. Ask the sub-groups to discuss:

What are the advantages of each type of question? What are the disadvantages of each type of question? When would you use each type of question?

Invite participants to share their conclusions.

Using the different types of question

Use the PowerPoint® presentation to summarize the advantages and disadvantages of the three types of question and when to use them.

Small group exercise – devising an opening question

Divide participants into sub-groups of four. Give each sub-group a copy of the behavioral indicators on page 24 and a copy of the candidate’s resume on page 26.

Ask the groups to devise an opening question that they might use to begin the process of uncovering evidence that the candidate displays each of the behaviors on page 25.

Invite each sub-group to share one of its questions. Invite feedback from the rest of the group as to the likely effect of each question.


P.R.O.B.E. for evidence

Use the PowerPoint® presentation to explain P.R.O.B.E. Refer to page 17 for information on this acronym.

Role-Play:

This is a chance for participants to practice putting together effective behavioral questions. Divide the group into pairs. Each pair will consist of an interviewer and a candidate. These roles will be reversed in the second half of the exercise.

Each participant thinks of a real job that he or she might recruit for. Each participant needs to spend ten minutes preparing behavioral questions. He or she then interviews the “candidate” to obtain evidence that this person does or does not display the behaviors sought. Then reverse roles.

During the interviews walk around the room, listening to what is being said they and offering feedback where needed.

If your organization has a competency framework, use this as the basis for participants to prepare questions. If not, use the behavioral indicators from page 24. Or, prepare behavioral indicators in advance of the session.

Conclusion

Highlight the main points from the session. Administer the post-test on page 27 to evaluate what has been learned.


A short guide to behavior-based interviewing

What is behavior-based interviewing?

Behavior-based interviewing is a technique where questions are asked about past behaviors. It is effective because past behavior is the best indicator of how a person will behave in future.

It is different from biographical interviewing. Biographical interviewing also involves asking questions about the past and is what most interviewers actually do. In biographical interviewing, you ask questions like, “I see you worked in the engineering department for three years. What exactly did you do there?” You are likely to get an answer like this: “I worked on the design of the T54. My job was to take the specifications that the market research department had generated and convert this into an engineering specification”. This is useful information and you will often need to ask questions like this before you can ask behavioral questions.

A behavioral question goes deeper. It focuses on one specific incident, sometimes called a critical incident, and probes to find out how the individual behaved during that incident. An example of behavioral question is: “Can you tell me about a time during your work on the T54 when you realized that you were not going to complete a drawing on time? (“Yes”). What did you do?”

It is also different from hypothetical interviewing. Hypothetical questions are like this: “What would you do if you realized that you were not going to complete a drawing on time?” This can be useful for understanding someone’s thought process but not a reliable indicator of what someone will actually do. Someone could tell you that if the building caught fire they would rush repeatedly into the flames, rescuing children, cats and old people. What is more, they might believe it. But this does not mean that in an actual fire they would not clamber over children and elderly ladies in their rush to escape.

Behavior-based interviewing is also called competency-based interviewing or critical-incident interviewing.

What is behavior and why does it matter?

When psychologists talk about behavior, they refer to all the responses that a person has to a stimulus. This is different from the everyday use of the word, where we tend to talk about children behaving badly, for example. The psychologist means everything that goes on in and from a person in a particular situation. This includes what they think, what they feel and what they do. Being scared is behavior and laughing is behavior. So is hitting someone.

At work, we are interested in the way people do their job. This is observable behavior. All sorts of things might be going on beneath the surface but the part we can see is the way they do their job.

When we recruit somebody, we want to know how he or she will approach his or her job. In other words, what their observable behavior will be. The best indicator of this is their past behavior and that is why the behavior-based interviewer is interested in it.

A person’s observable behavior is like the tip of an iceberg. It is the visible portion of a complicated mass of things we cannot see. These are things like how the person feels and what he or she thinks, what motivates the person and what traits they have. In other words, behavior stems from the underlying personality, in all its complexity, of the person concerned.


We cannot begin to understand all that is going on beneath the surface. Much of it is not even known to the person himself or herself. But behavior is remarkably consistent. If a person is scared atop the Sears Tower, they are likely to be scared on the 82nd floor of the Empire State building. We do not have to understand everything that is going on inside to make predictions about how someone will behave in a given situation.

Does this mean we cannot change our behavior? Certainly not. But we do have natural characteristics, which tend to assert themselves again and again. We can adjust these and in some cases, such as overcoming a fear, we can change our behavior, but past behavior remains the best, though not a perfect, indicator of how we will behave in future.

The Advantages of behavior-based interviewing

Behavior-based Interviewing can be summed up as:

Effective Objective Transparent Legal

EffectiveThe key advantage is that it gets better results. Behavior-based questions find out how people have actually behaved in the past. This gives you a much better indication of how they will behave in the future.

ObjectiveBehavior-based interviews focus on what someone actually did and how they actually did it. They then compare the way someone has actually behaved with behaviors that the interviewer is looking for. This is an objective process, unaffected by subjective feelings that the interviewer may have.

TransparentBehavior-based interviewers are open with candidates about the skills they’re seeking. This enables them to involve the candidate in helping them in the process. Otherwise, an interviewer might miss a good candidate because the candidate fails to recognize what the interviewer is looking for and not provide the right information. The candidate can’t tell the interviewers “what they want to hear” because he or she will be describing actual events. The candidate would have to construct a complicated lie very quickly to do this, and it would certainly not hold up as questions proceeded.

LegalBehavior-based interviews are fair because they are objective. The competencies sought are openly described and all candidates are given an equal chance to demonstrate that they have those competencies. Provided every candidate is asked for evidence of the same competencies, and provided those competencies are genuinely necessary to the performance of the job, you will not discriminate by asking behavior-based questions. Furthermore, if you are challenged over your reasons for a recruitment choice, you will have objective evidence to substantiate your decision.


The relationship with competencies

We use the word competency to mean a cluster of behaviors. Examples of competencies are: leadership, problem-solving and analysis, delivering results, managing change, decision-making, teamwork, customer service, developing people, communicating and influencing, strategic thinking, networking, interpersonal skills and communication skills. These are all examples of competencies that have been identified by organizations as being important to their success. In addition to these “core” competencies which could apply to any person in an organization, we have technical or specialist competencies such as: finance and accounting, IT, legal, geological, medical, safety and so forth.

You can describe each competency in terms of specific behaviors. Take communication as an example. What does a good communicator actually do? Here is a list of observable behaviors from a real competency framework:

Conveys information clearly and concisely, verbally and in writing Comprehends verbal and non-verbal communication in a wide variety of cultural contexts Identifies and exploits the optimum communications media for various messages and

audiences Deals with conflicts in a manner that achieves mutually acceptable outcomes Contributes to meetings in a manner that assists in moving the agenda forward

These behaviors would look different at different levels in an organization. A chief executive might be expected to be able to communicate clearly and concisely to a Boardroom Committee while a mid-level supervisor might be expected to communicate a task to his or her team. A proper competency framework will involve descriptions of what each behavior will look like at different levels.

Decide what competencies you are looking for

If your organization has already developed a competency framework, it will have defined the competencies for you and described them in terms of observable behaviors. You will need to examine the job vacancy carefully to decide which competencies apply to it.

If your organization does not have a competency framework, you will have to analyze the job vacancy yourself and decide what competencies are needed. This is a very useful exercise, forcing you to be clear about what you are looking for in a candidate.

As well as considering the competencies you want, consider those competencies that you do not want to see in a successful candidate. For example, you might consider that risk-taking a competency that you do not want to see to a high degree in a trustee of a pension fund.

Decide which competencies really matter to the job and draw up a list of these. You won’t want more than five general competencies plus three specialist ones.

Describe these in terms of observable behaviors

This may have been done for you by your organization. If not, ask yourself what success in each competency would look like. Remember to ask what a good candidate would actually do for each competency. Paige says in the DVD that a good problem solver solves problems. Of course! But what does that involve? James asks her to break it down into component behaviors: identifies problems clearly; generates a number of possible solutions; evaluates the solutions effectively;


chooses the best solution; implements the chosen solution effectively. Never go into an interview looking for a “good leader”. Work out what a “good leader” does and then look for someone who can demonstrate that they do it.

Preparing the questions

Don’t over-prepare. Some interviewers prepare dozens of questions and then find that they are not listening to the replies because they are busy finding their way around their list of questions. Trust yourself to think of questions on the fly. If you are listening carefully, it will be obvious what to ask next, especially if you are using the P.R.O.B.E technique that we describe later.

Behavior-based questions are very powerful but they are not good at building up a general picture of the candidate. You need biographical questions for this so it is a good idea to prepare both biographical and behavior-based questions.

Prepare some icebreakers, too. Things like: Did you find us all right? Have you been to here before? Have you been enjoying this lovely weather?

Your choice of biographical questions will be influenced by the candidate’s resume” and other information that you have about the candidate. You won’t want to ask questions that have been answered in the candidate’s resume” but you can prepare useful questions such as, “will you tell me about your time at Smith and Jones?” Use these biographical questions to build up an overview of what the candidate has done and what he or she has achieved.

The most important questions will be the behavior-basedones. Think about the competencies that you have identified and prepare an open question for each one. Judi did this with Randall in the DVD when she wanted to know about His leadership competency: “Randall, one of the things this job would involve would be leadership. Can you give us an example of when you have been in charge?” Here are a couple of examples you could use if the competency was managing conflict: “Tell me about a time when you had to resolve a conflict at work between two people?” or “Tell me about a time when you found yourself in conflict with someone?”

Asking the questions - P.R.O.B.E. (see acronym page 17)

An important characteristic of behavior-based questioning is transparency. There is no point in being secretive about what you are trying to find out, as you need the candidate’s help in finding evidence of competencies. Tell the candidate what you are looking for: “One of the things we are looking for is someone who works well in isolation. Can you describe a situation in which you worked for a week or more without contact with your line manager?” This maximizes the chances that the candidate will give you useful and relevant information. Don’t worry that the candidate will cheat if he or she knows what you are looking for. It would be extremely difficult to invent a convincing story that was not true and it would almost certainly break down under questioning.

The most important aspect of questioning is to listen to the answers and to respond to what you are hearing. Don’t proceed through a rigid list of questions regardless of what the candidate says. Have in front of you a list of the specific behaviors that you are looking for under each competency. Use your prepared open question to get the candidate talking about the competency and then look for evidence of the specific behaviors that you identified.


P.R.O.B.E stands for:

Problem (or position)RoleObjectiveBehaviorEffect

You start with your open question to get the candidate talking about the competency in which you are interested: “We are looking for a person who handles conflict effectively. Can you describe a situation in which you had to deal with a conflict between your colleagues?” When the candidate begins to describe the incident, listen to make sure that you understand:

What was the position or problem that the candidate had to deal with?What was the candidate’s role? Was he or she in charge? A subordinate? A peer?What was the objective that the candidate was trying to achieve?What did the candidate actually do? i.e. how did he or she behave?What was the effect of that behavior?

You will find that you know what questions to ask if you keep PROBE in mind as the candidate talks. Don’t ask questions about every letter in PROBE just for the sake of it and don’t feel you have to ask them in the “right” order. You will look pretty silly if the candidate says, “I faced this situation just after I took charge of the team” and you say, “and what was your role in the team?” Likewise, if the candidate says, “I told them to stop fighting” and you say, “and what was your objective in telling them to stop fighting?” Look for the parts of PROBE that have not been covered in the candidate’s answer and then ask follow-up questions to get these clear. If you cannot explain what the problem was, what the candidate’s role was, what he or she was trying to achieve, what he or she did and what happened as a result then you have not asked the right questions or you have not listened to the answers.

Try to be neutral in your reaction to questions. “Thank you for that. That was very helpful” is much better than, “Wow! That was amazing. I can’t believe you did all that in two weeks”. Remember, your impression during the interview may well not be your final reaction. There is too much going on during an interview for you to make the kind of cool assessment that you need to make. This will come later when you review your notes and examine the evidence that you have collected. If you let your immediate reaction show too strongly you run the risk of misleading the candidate as to how well, or badly, the interview is going.

Taking notes

This is the secret of a successful interview. You can’t fairly evaluate a candidate’s performance without good notes.

Your purpose is to make sure that you can consider each of the competencies that you are looking for and then state clearly what evidence you have found that the candidate either does or does not behave in the way that you require. This means having a well-structured system of taking notes such as we have illustrated on page 25.

You will need plenty of paper! You will certainly need at least one sheet for each competency that you are interested in. A simple approach is to have a sheet with the appropriate competency as a heading. Make two columns and in the left column, list the specific behaviors that you are looking for. In the right column, note key words and phrases that will remind you of things that the candidate said that demonstrated that they display the behaviors in the left column.


After the interview, review each competency in turn and record your reasons for believing that the candidate demonstrates the behavior that you are looking for by reference to what they actually said.

Working in a group

Working in a group has clear advantages over interviewing alone. Colleagues may pick up things that you might miss, and vice-versa. You can discuss your findings after the interview and challenge each other to justify your conclusions. You can spread the workload so that one person is concentrating on making notes while another is following a line of questioning and you can make one person responsible for watching the time.

Working as a panel also involves discipline. You need to agree to clear rules for asking questions. One approach would be to allocate different competencies to different interviewers and agree that no one will interrupt another panelist until that panelist is finished.

Giving feedback

You are going to end up rejecting candidates. Traditionally, organizations have sent a short note saying that, unfortunately, they are unable to offer the candidate this particular job but they wish the candidate well in future.

The modern approach is to offer the candidates feedback, should they wish it. Many candidates do wish it because it gives them a chance to appraise their own performance and possibly take action to remove a weakness.

Provided you have asked fair and open questions about behavior you’re seeking provided you have kept comprehensive notes, you should have no problem in giving helpful feedback. Don’t tell the candidate that there is something wrong with them – you have no right to do that and you are not in a position to do so.

Express your feedback in terms of a comparison between what you were looking for and the evidence that you found. Refer to what the candidate said when you do this. Say something like, “We were looking for an individual who solves problems under pressure. You told us about the time when you were involved in project x and you said that you did such-and-such which did not indicate to us that you did solve problems under pressure in the way that our job requires”.

This will be very useful to the candidate who may either reappraise the kind of job that he or she is looking for, or re-examine his or her work experience for better evidence to use in future interviews.


Analysis of interviews

Interview of Eric by David and Paige

Overview

This interview was largely biographical. David and Paige found out a lot about what Eric had done in his previous job but they did not find out a great deal about how he had done it. They asked some perfectly sensible and effective questions and it is important to understand that Eric did not lie or intend to mislead. So what went wrong?

It turned out that there was a mismatch between Eric and the vacancy at Paige’s company. An individual’s performance does not occur in isolation. It results from the interplay of three factors:

the individual the situation the organizational environment.

It is quite possible for an individual to be very effective in one environment and situation and much less so in another. When an individual changes jobs the situation and the environment both change. You cannot assume that because an individual was effective in one situation and environment that he or she will be equally effective in another situation and environment. It is entirely possible that Eric was very effective in his previous job. David and Paige had to try to gauge how effective he would be in this job. Just knowing that he was effective previously does not tell them this. They needed to find out how he actually behaved in order to judge how that behavior will work in their environment and situation.

Paige and David clearly liked Eric. He had been part of a success story and this impressed them. They were also under pressure themselves to find a good person to get them out of the problem that they faced at work. The combination of these factors led them to stop listening critically to what Eric was saying and to hear what they wanted to hear.

The questions

Paige’s questions were perfectly sensible. They were good, open questions that were effective in getting Eric talking:

What did you do there? How did you go about it?

The problem lay in the lack of thorough follow up. Paige allowed Eric to explain what “we” did. In effect, Paige was interviewing Eric’s previous employer. She should have probed much more deeply to find out specifically what Eric did, why he did it that way and what the effect was of what he did.

David noticed that Eric had said that he was the assistant team leader and so David sensibly asked him to explain the different roles that he and his boss had. However, David wanted to hear that Eric achieved results on his own initiative and so he asked the leading question:

So you were the driving force behind all this change?Which can only be answered “yes”.


Interview of Randall by Judi and James

Overview

The contrast here is between Judi and James. Judi started well but then moved into a series of ineffectual biographical and hypothetical questions. James then took over in the middle of Judi’s questions. (This is not recommended and you may wish to discuss whether he should have done this. One of the cardinal rules of panel interviewing is that interviewers don’t interrupt each other. They should have a clear plan for who is going to speak at which time and they should have a system for handing over to each other.)

However, James did interrupt and take over and his reason for doing so was that he could see that Judi was going to move on without learning anything about how Randall behaved at work. Once he had taken over, he began to ask fairly searching behavior-based questions that elicited useful information about how Randall worked. He did this by asking about a specific occasion when Randall’s boss had been away. This is the essence of behavior-based interviewing: he did not ask what Randall might do or would do; he asked him what he actually did on a specific occasion.

The questions

Judi began with an excellent behavioral question:

Can you give us an example of when you have been in charge?

Instead of following this through with probing questions about what happened on that occasion, she allowed Randall to give a general answer:

When my manager is away, I am in charge of the group.

Judi should have asked:

Can you tell me about a particular time when this happened?

And then followed up with a series of questions designed to find out how Randall approached the situation. She actually followed Randall’s lead into a general discussion. The information that Judi obtained – that Randall managed six people – was useful but not enough.

Judi then asked a classic ineffective question:

What problems do you find in being in charge?

This is one of those questions like, “what are your weaknesses?” that puts the candidate in control. The well-prepared candidate can give you any answer he or she chooses to this kind of general question. Obviously, Randall was going to say that he found no problems in being in charge. Judi then continued with the entirely useless:

So you wouldn’t have any problems in leading a customer service team here?

Can you imagine any candidate saying, “Oh, gosh. No, I doubt if I could do that. I’m worthless at leading a team”?

Judi then went on to ask:


Let’s go on and see how you would deal with a customer who had a problem.

This suggests that she had been trained in behavior-based interviewing but had not really got the hang of it. Her opening question in the interview suggests the same thing. This is quite common. Many people think that they are asking behavior-based questions because they start by asking about a specific situation. The opening question is not enough. You have to keep asking questions about what a person actually did and what the effect of that behavior actually was.

James did this well.

He was also transparent. He said openly:

We are looking for someone who sets priorities effectively.

This gave a context for his questions and helped Randall to present useful information. As it turned out, Randall did not set the priorities. This is a powerful example of how behavioral questions can work. If James had asked, “how would you go about deciding what is important?” he might have got a perfectly convincing answer. But by asking how Randall actually did it on this occasion, he uncovered the fact that he did not actually set priorities in his present job. Incidentally, this does not mean that he cannot do so. It only means that we cannot tell. It might well make sense to employ him but to watch carefully to see how he fared in this area and to be prepared to intervene to help him develop this behavior.

James asked:

How did you handle those that were unhappy with what they’d been asked to do?

Again, this was a strong behavioral question about a specific event. Randall could only answer it by describing what he actually did and this provided good evidence that he allocated tasks effectively and gained commitment to getting them done.


Interview of Shawna by James and Paige

Overview

Paige was transparent about what she was looking for:

Shawna, we’re looking for someone who solves problems.

And she asked questions about a specific situation. She also used the power of you effectively to bring Shawna back to what she did:

When you say, “we”, can you tell me what you did?

She asked a series of probing questions to give her detailed information about how Shawna actually behaved:

What did you do? How did you fix it? What did you come up with? What did you do if the guy from IT came up with the solution?

Taken together, these all gave her good evidence of how Shawna actually behaved at work.

James was taking notes while Paige was asking the questions. This was a sensible division of labor as it can be difficult to listen to what someone is saying if you are taking notes and asking questions. We saw in the review of the interview that James was using a structured form on which he had recorded specific evidence of specific behaviors. This form is reproduced on page 25.

The questions

Paige asked a good opening question to get Shawna talking about a specific event:

Can you think of a problem that you’ve solved recently at work? One where you’re pleased with the way you did it?

She then directed her towards talking about how she actually behaved. Shawna answered her opening question by describing what her company had done: “We organized a new security protocol to bring us in line”. Paige saw the danger of interviewing her previous employer and re-focused her with:

When you say, “we” can you tell me what you did?

This uncovered the fact that she personally had identified the problem. This was good evidence of what she was looking for – someone who identified problems. She asked:

What did you discover, exactly?

And this caused Shawna to give a clear description of the problem. This was good evidence that she identified problems clearly, which was exactly what Paige was looking for. If she had given an unclear reply, this would have been counter-evidence of this behavior.

Paige further clarified her description with:


What did you have?

And she then confirmed that Shawna had personally identified the problem with:

How did you uncover the problem?

She then moved on to investigate how Shawna went about solving the problem. Here she was disappointed. She was looking for evidence that she generated a range of solutions and evaluated them. She found no evidence of this because it turned out that another individual had put forward the solution. She confirmed this conclusion with:

So you weren’t personally involved in developing the solution to the problem?

This was a good example of using a leading question appropriately. Paige had already ascertained this and used a leading question to confirm her understanding.

In this part of the interview, Paige and James found good evidence that Shawna identified problems clearly and, later, that she implemented a solution. They were left with gaps in their evidence concerning whether she generated solutions and evaluated them. They found some evidence of this elsewhere in the interview and decided to hire Shawna.

They would have been wise to watch her performance carefully in the area of problem solving and to be ready to help her develop her abilities in this area if they had detected any weakness.


Behavioral indicators

Leadership Acts on own initiative in response to broad planning guidelines

Inspires and motivates others in complex and rapidly-changing circumstances

Changes behavior appropriate to differing situations

Contributes to achievement of goals at own leadership level

Takes risks and succeeds on majority of occasions

Recognize and rewards achievements Supports individual development Plans and assigns tasks up to two weeks ahead.

.

Communication

Conveys information clearly and concisely, verbally and in writing Comprehends verbal and non-verbal communications in a wide variety of cultural

contexts

Identifies and exploits the optimum communications media for various messages and audiences

Deals with conflict in a manner that achieves mutually acceptable outcomes

Contributes to meetings in a manner that assists in moving the agenda forward.

Teamwork / Working together

Works cooperatively and effectively with others in a complex political and social environment

Contributes to the resolution of differences of opinion, values, and priorities

Contributes to the achievement of team goals

Can explain common team goals and how they contribute to organizational objectives

Takes responsibility for own work and delivering results.


James’s notes on Shawna

Candidate: Shawna

Date of interview: 31st October

Competency: Problem solving

Behavior Evidence

Identifies problems clearly

Ppa regDataGovt. siteSecurityaccess

Generates possible solutions

?Guy from IT?PTOs school

Evaluates solutions clearly in terms of costs and benefits

Nothing from workAd book with local businessFootball club

Implements chosen solutions effectively

Team meetingShort-term targetsTraining session

Works with minimal supervision

Boss away“trust you to get it done”Colleague sought helpSupport from team“I like it that way” comment


Candidate’s resume’

Kim Smith-Jones

Address: 23 The Road, Anytown, Anywhere, 12345Telephone: 012-345-6789e-mail: [email protected]

StatementI have had substantial experience of the kind of work that you are looking for. I possess excellent communication skills, and I am a go-getting self-starter equally at home in the private and public sectors. I feel that my experience uniquely qualifies me to undertake any job that you might have and believe that I can add value to any organization that employs me.

Employment history

Feb XX to date General administrator, Oldsocks and DarnMaintained all filesKept all accounts up to date using RAGE softwareResponsible for training new hiresDealt with customers and suppliersAttended all meetingsProofread minutes of meetings

March XY to Feb XX Senior engineer, Calibration SystemsKept all specifications up to dateResponsible for design drawingsTrained designers in use of Pac softwareCarried out stress tests on wing models

Voluntary work I spend two evenings each week organizing social events for SPENDTHRIFT, a charity that helps people who are persistently in debt. This involves me in organizing venues, entertainment and food as well as travel and logistics.

Interests: Walking, Reading, Movies, Polish history, Motorcycle restoration.


mailto:[email protected]

Post-course test

For each of the following questions, state whether they are behavioral, biographical or neither by checking the appropriate box.

Question Behavioral Biographical NeitherWhat was your job there? How many people worked for you? How would you tackle a shortfall? Why did you change jobs? How did you react to his feedback? Why did you confront him at that point? Did you enjoy working there? Tell me how you approached that breakdown? What are your strengths? What did you say next?

For each of the following questions, state which kind of question would best give you the information that you are seeking by checking the appropriate box.

Desired information Behavioral Biographical HypotheticalA general picture of a candidate’s career

Whether a candidate can solve a work-related problem not encountered before

How a candidate responds to pressure

How a candidate organizes time

Whether a candidate handles conflict effectively

What tasks a candidate dislikes

Whether a candidate multi-tasks effectively

How a candidate will react to an angry customer.

You have identified the following behavior as being desirable: Contributes to meetings in a manner that assists in moving the agenda forward.

Write down what you would say in order to begin the process of looking for evidence that a candidate exhibits this behavior:

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Thank you. That is the end of the test. Please fill in:

Your name ______________________ Date_______________________________


Post-course test answers

For each of the following questions, state whether they are behavioral, biographical or neither by ticking the appropriate box.

Question Behavioral Biographical NeitherWhat was your job there? How many people worked for you? How would you tackle a shortfall? Why did you change jobs? How did you react to his feedback? Why did you confront him at that point? Did you enjoy working there? Tell me how you approached that breakdown? What are your strengths? What did you say next?

For each of the following questions, state which kind of question would best give you the information that you are seeking by ticking the appropriate box.

Desired information Behavioral Biographical HypotheticalA general picture of a candidate’s career

Whether a candidate can solve a work-related problem she has not encountered before

How a candidate responds to pressure

How a candidate organizes his time

Whether a candidate handles conflict effectively

What tasks a candidate dislikes

Whether a candidate multi-tasks effectively

How a candidate will react to an angry customer.

You have identified the following behavior as being desirable: Contributes to meetings in a manner that assists in moving the agenda forward.

Write down what you would say in order to begin the process of looking for evidence that a candidate exhibits this behavior.

You are looking for two elements: a transparent statement such as, “We are looking for someone who contributes to meetings and helps move the agenda along.” And a good open question, such as, “Can you think of a meeting that you attended which was getting bogged down and which you helped to move forward? What happened?”

SCORING For full marks, both elements should be present. Mark the transparent element out of 3 and mark the question out of 4, using your judgement. This makes a total of 7 for this question. Each correct multiple-choice question counts as 1 giving a maximum total of 25.


a question of evidence - user's guide - telephone doctor€¦ · web viewuse the...

Documents