psychometrics 101: a guide to choosing and applying the

Psychometrics 101: a guide to choosing and applying the right option

Love them or loath them, Psychometrics are to be here to stay and the industry is growing rapidly. Surveys conducted by the Chartered Institute of Personnel and Development (CIPD) show that within The Times Top 250 organisations 80% use tests of ability and 50% use Personality assessments (cited by The Psychometrics Forum, 2013). In the US, Psychometric assessments have been found to be used by 97% of Fortune 500 companies (EMetrixx, 2013).

One of the prime reasons for this surge in Psychometric test usage is the evidence base which shows them to be strong predictors of job performance when used correctly (Barrick & Mount, 2001; Hunter & Schmidt, 1998).

We regularly help clients make the best use of Psychometric profiling and we have put this guide together to answer some of the questions we are asked most frequently, including what tools to use in different situations, when to use them and how to go about it.

A complex range of options – just where do you start?

There are hundreds of different tools and instruments on the market, each with varying applications and uses.

It can be a daunting prospect knowing where to start: when to use psychometrics, how to apply them, which tool to use and how to analyse the results.

Psychometrics: how to make the right choices

“The core purpose behind the use of Psychometrics is to measure the differences that exist between individuals. Simply put, Psychometric means ‘measuring the mind’, and they are typically used to objectively measure quite inherently complex psychological phenomena e.g. personality and intelligence.”

“Love them or loath them, Psychometrics are to be here to stay and the industry is growing rapidly.”

Email [email protected] Call +44 (0)20 8371 7011 Visit www.jsbonline.com

When should I use Psychometrics and when

should I avoid them?

Aren’t Psychometrics

just for recruitment?Which tools

are suitable for different uses?

What can I do

to measure the

effectiveness of

learning?

How to I interpret and use

the results of my tests?

How do I choose the right

tool for the right job?

I need to benchmark performance and capability

– how can I do this?

Do any of these questions sound familiar?

What are the common pitfalls to

avoid when using Psychometrics?

This guide will help you answer these questions and provide you with enough of a level of understanding to enable you to build Psychometrics into your recruitment and development activity.

Visit www.jsbonline.com Email [email protected] Call +44 (0)20 8371 7011

Psychometrics 101:how to make the right choices


Reports and outcomes from psychometric assessments can help:

■■ Individuals to better understand themselves■■ Organisations to gain objective results and to compare individuals fairly■■ Organisations to ensure fully informed decisions are made in recruitment and promotion

To Recruit: To Engage: To Coach: To Achieve:

Attract quality candidates Motivate and inspire Performance reviews High flyers

Screen candidates fairly Resolve conflict Mentor Productivity and efficiency

Selection Boost morale Develop leadership skills Optimal team performance

Assessment centre use Creative innovation Appraisals A leadership pipeline

Improve basis for recruitment Bolster performance Teamwork Organisational redesign

Position talent Foster loyaltyDevelop communication and relational skills

Improved people management skills

Promotion assessment Restructure roles Develop career objectives Reenergised employees

The core purpose behind the use of Psychometrics is to measure the differences that exist between individuals. Simply put, Psychometric means ‘measuring the mind’, and they are typically used to objectively measure quite inherently complex psychological phenomena e.g. personality and intelligence.

Psychometrics in an organisational context are essentially occupational tests or questionnaires. However, the nature of the human characteristic that is being measured dictates the nature of the ‘test’. For example, in the case of intelligence or aptitude there is a degree of right or wrong involved.

However in the case of personality, there are no real right or wrong answers but a stronger or lesser preference towards certain behaviours or presence of certain traits. Psychometrics can be used in a range of contexts within an organisational life cycle, from recruitment to training and development, to managing underperformers.

What are Psychometrics?




Ability Psychometrics (Maximal Performance)

Ability Psychometrics are broadly categorised into three types; intelligence, aptitude and attainment assessments. Intelligence Psychometrics can be designed to measure general intelligence (defined as ‘g’), namely fluid and crystallised intelligence or more specific, narrow intelligences such as numerical or verbal reasoning. In contrast, attainment tests assess learning that has already occurred such as university exams or end of training exams. Lastly, aptitude tests assess the ability to acquire and use knowledge and tend to be job role specific.

Personality Profiles, Portraits, Inventories and Psychometrics (Typical Performance)

Academics as well as practitioners have (and probably always will) argue about the definition of personality. However, across the many varied conceptualisations of personality, one aspect seems to be agreed upon. And this is the consensus that personality is made up of a combination of underlying personal characteristics, which are relatively stable across different times and situations. Hence, these tools are used to predict an individual’s typical performance and behavioural preferences.

Personality types vs. personality traits

The Psychometric approach to personality (as opposed to the Psychoanalytic, Humanistic, Social Learning and many other schools of thought) assumes that human attributes can be quantitatively assessed and measured using techniques such as questionnaires. An important distinction to bear in mind is the type vs. trait approach in personality.

The former is ideographic in nature in that it divides individuals into discrete categories (or types) which have their own set of characteristics. This typically occurs in opposites i.e. an individual is either an extravert or an introvert. A combination of these types then gives rise to a complex human Personality – such as an ENFP in the Myers-Briggs Type Inventory (MBTI).

In contrast, the trait approach to Personality emphasises the continuum of traits. A trait is described as an underlying characteristic of an individual’s personality. There are two extremes of a trait, hence a scale concept is introduced here. The degree to which an individual possesses a trait can then be compared to a general population norm or indeed within teams/organisations, so there is a relative comparison of personality traits. Commonly used trait tools include the OPQ and NEO-PI-R.

There are a number of fundamental attributes an assessment must possess if it is to provide valid results.

To be psychometrically sound a test must be:

■■ Objective The results obtained must not be influenced by the administrator’s personal characteristics or other irrelevant factors.

■■ Standardised The test is administered and scored according to standard procedures, and people’s scores are compared to known standards.

■■ Reliable The test measures in a consistent way. The potential error is small and is quantifiable.

■■ Valid The test measures the characteristics which it sets out to measure. A test used to select a job applicant should predict job performance and not some other skill.

■■ Discriminating The test should show clear differences between individuals on the behaviour being tested.

Different types of assessmentRequirements of a valid Psychometric test




How and when to use Psychometrics?

Too much reliance:

Assessments are a useful part of projects and interventions but it is not helpful to rely too much on them. Even when used well assessments are best deployed in combination with other techniques such as 1:1 coaching or with competency interviewing when part of a selection process.

Inaccurate interpretation:

Inexpert or – where relevant – unaccredited users may make inaccurate or incomplete interpretations of assessment results. This does a disservice to the individual assessment taker, the client organisation and also risks damaging the credibility of the assessment.

Discriminatory usage:

Another potential pitfall of using assessments is that some of them inadvertently may discriminate unfairly against some test-takers – specifically on the grounds of cultural differences or English language skills (factors which are irrelevant to the reason for using the assessment). It is therefore important to take these factors into consideration when choosing the assessment tool to use.

Using assessment and profiling tools can have many benefits if best practice is followed. The factors below represent common misuses of Psychometrics which undermine their use and damage the validity of the results they produce.

Selection and Assessment

Psychometrics are becoming more and more commonly used in recruitment, particularly for graduate roles.

Hence, it is increasingly important to ensure that they are being used in the right way at the right time. Of paramount importance is the use of Psychometrics alongside other methods for selection i.e. interviews, assessments centres and work sampling in order to increase the predictive validity of your methods of selection.

Predictive validity represents the correlation between the test scores and the degree of success in the subsequent job. And using a psychometric along with other selection methods improves incremental validity, which represents the added value that a specific selection tool gives you during the selection process, explaining the unique contribution of that measure.

Best practice has been outlined in recent research by Campion et al (2011) who encourage the use of job analysis in order to construct a competency framework that is in line with organisational objectives, visions and mission statements.

This framework then becomes the basis for which we can decide whether certain Psychometrics can be used to measure these competencies. The ends must be thought about before the means. Therefore it is crucial to assess whether Psychometrics can actually help you measure the competencies which have been identified as crucial for a particular role.

The next decision which has to be made is choosing a particular Psychometric out of the huge pool of tools that exist. Best practice would dictate that validity, reliability and fairness criteria are considered, which generally means reading the technical manual of a Psychometric tool to judge its quality. However, the good folks at the British Psychological Society’s Psychological Testing Centre (PTC) have reviewed many of the tools out there in the market to help organisations make well-informed choices for their needs.

Potential pitfalls of using Psychometrics

Job Analysis/Job description

Competency framework

Selection of appropriate Psychometric designed to assess competencies




Before choosing the appropriate tool, you should always start by asking the question ‘What is the problem you are trying to solve? ’ Detailed below are some of the wide ranging uses of psychometrics within an organisational setting.

Learning and Development

Choosing the right tool

Similar considerations apply when Psychometrics are used in a learning and development context. The importance of job analysis and Psychometric validity and reliability are key concerns when deciding which tools to use for training, career coaching and leadership development.

Typically, personality measurements are used more widely in this context than an assessment and selection context. Personality profiling adds value in the career coaching conversation as it identifies and raises awareness about potential weaknesses which could act as barriers to development.

As well as this, personality profiling can define real strengths which, if used to map against job roles, can help individuals remain satisfied in their choice of career and to develop in ways that match their personal motivations and preferences.

360 Degree Feedback

Coaching

Performance Management

Induction

Strategy

Engagement & Motivation

Management Development

Team Development

Needs Analysis

Bench MarkingCareer

Development

Conflict Management

Organisational Development

& Culture

Diversity Awareness

Assessment & Selection

Client Relationship

Development

Creativity

Relationship Management

Business Restructuring

Leadership

Sales Negotiation

Mediation

Strengths

Communication

Retention

Change Management

Emotional Intelligence

Personal Development

Work Style Assessment

Succession Planning




Having chosen the right Psychometric for your organisational need, and administered it in line with best practice, the next step is to interpret and use the results. Psychometric test users must be well aware of the intricacies of interpreting results and the importance of sensitive and accurate feedback.

In the case of intelligence tests, it’s crucial to stress the context in which results are being fed back. Generally speaking, results are reported as percentiles in this case, comparing candidate scores to a norm group rather than absolutes figures. In the case of personality this becomes even more complex because of the lack of right/wrong answers. There may well be elements of a personality report that individuals disagree with – which is to be expected, as all instruments have elements of measurement error.

Crucially, it must be taken into consideration that the accuracy of results are based on the information that the candidate has provided and is dependent as much on their openness and honesty as the validity and reliability of the tool.

In this context language is very important, as when results are interpreted and fed back to candidates care must be taken to avoid overstretching and making too strong of an assertion based on the results. Remember, these are measures of typical performance. Hence, these tools describe a tendency to behave in certain ways as opposed to a behavioural crystal ball which predicts a candidate’s future.

When using particularly complex and technical Psychometrics it is required that they be interpreted by trained/accredited Psychometricians to avoid any mistakes, bad practice and ultimate misuse of Psychometrics. Finally, returning to a point made earlier, while Psychometrics can add invaluable richness to an assessment of an individual, they are not robust enough to be used independently. Therefore, they should be used in conjunction with other methods i.e. one-to-one coaching, 360 degree feedback and interviews.

■■ Commerciality and mainstreaming of Psychometrics The majority of Times Top 250 and fortune 500 companies use Psychometrics. The need for SME’s to do the same is even greater given the considerable costs of recruitment errors and training and development investment.

■■ Wide range of use Psychometrics can be used in a number of contexts from recruitment, to engagement to talent retention and even managing underperformers.

■■ Addressing your need Psychometrics can be used to address a range of organisational needs, such as leadership development, team-working, on-boarding, strategic planning and appraisal (360-degree feedback). It is important to understand why Psychometrics are needed and what value they can add in each instance.

■■ Quality and best practice It is also crucial to ensure that Psychometrics meet certain standards of quality and that their use is in line with best practice.

Summary Interpreting Psychometrics

“Psychometrics are not robust enough to be used independently.”

“There may well be elements of a personality report that individuals disagree with – which is to be expected, as all instruments have elements of measurement error.”

“Crucially, it must be taken into consideration that the accuracy of results are based on the information that the candidate has provided.”




We are learning and organisational development consultants. We help people and organisations to build and sustain outstanding performance through our learning, development and consulting services, plus a range of open courses, conferences and seminars.

We work with over 30 different personality Psychometrics, behavioural/motivational profiles and 360 degree feedback tools, which can be used across a range of contexts. As well as this, a number of our consultants are Occupational Test User qualified (Level A & B).

About JSB

Next Steps

If Psychometric test use in your organisation interests you, get in touch with us to find out how we can work together to utilise the power of Psychometrics to add value in your organisation. We have a number of case studies where Psychometrics have helped our clients address their organisational needs.

To request this information or find out more about how we can help you, visit www.jsbonline.com or call us on +44 (0)20 8371 7011.

References

Barrick, M. R, Mount, M. K. & Judge, T. A. (2001) Personality and Performance at the Beginning of the New Millennium: What Do We Know and Where Do We Go Next? International journal of selection and assessment, 9, 1/2, 9-30.

Campion, M. A, Fink, A. A, Ruggerberg, B. J., Carr, L., Phillips, G. M. & Ogman, R. B (2011). Doing competencies well: Best practice in competency modelling. Personnel Psychology, 64, 225-262

Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274.

E-metrixx.com. 2013. E-METRIXX | Psychometric tools to help improve your profit and people performance. [online] Available at: http://www.e-metrixx.com/ [Accessed: 21 Oct 2013].

Psychometricsforum.org. 2013. Psychometrics – Home, Psychometric Practitioners providing Psychometric Knowledge and Psychometric Information. [online] Available at: http://www.psychometricsforum.org/ [Accessed: 21 Oct 2013].




psychometrics 101: a guide to choosing and applying the

Documents