promise and perils of predictive analytics

© 2016, Future of Talent Institute

Kevin WheelerTalent Acquisition Tech Conference

Austin | November 15-16, 2016

The Promise & Perils of Predictive Analytics

1


Challenge 1: Ambiguity

Recruiters are asked to find people with skills we’re never heard of, in unrealistic timeframes, in remote places.


Challenge 2: Complexity

We need to know how to find the signal in the noise. Desperatly these days!


Challenge 3: Friction

Our processes do not flow. Required information is often not easily accessible. There are choke points and ambiguities everywhere.


Challenge 4: Interaction

Communication is unclear. We need to know what to say. What is effective?


Challenge 5: Innovation

6

Old thinking and bureaucratic processes stifle innovation and change.


The Promise

7

MORE EFFECTIVE COMMUNICATION

FASTER PROCESSES

GREATER CANDIDATE

UNDERSTANDING

HIGHER QUALITY

HIRES

INCREASED INTERACTION

ENAGING CANDIDATE

SERVICE

PEOPLE ANALYTICS


Predictive Analytics Promises Answers

8

• Who are our top performers?

• When & how should we connect with them?

• What attracts them to our firm?

• Which assessment is more accurate?

• Which hires will be the most productive?

• What would increase our quality of hire?

• Which interview questions are most effective?

• What will our turnover rate be in the next quarter?


Descriptive & Predictive Analytics Compared

9

Descriptive Analytics Predictive Analytics

PurposeUnderstand the Past

Observe TrendsDiscuss

Gain InsightsMake Decisions

Take Action

Timeframe Past and Current Future

Metrics Type Lagging Leading

Data Used Raw/Tabulated Information

Data Type Structured Structured and Unstructured

BenefitsUnderstanding

EfficiencyInformation & Insights

Effectiveness


Predictive analytics uses algorithms, machine learning, statistical analysis, sentiment analysis, semantic analysis, and other complex methods to provide insight.

But there are challenges and many things to lead you astray. . .

10


Disruptive Technologies

11

Internet

Social

Mobile

Cloud

Big Data - Analytics

Technology Foundation

Trends & Innovations

Internet of Things

Robotics

Disruptive Scenarios

Passive candidate assessment

Algorithms Automate Recruiting

Intelligent Personal AgentUltrasonic Tracking

Predictive Analytics DNA Analysis/Assessment

Virtual/Augmented Reality

ChatbotsBiometric Assessment

Life Span

Blockchain

Supply Chains

Contingent Workers

Climate Change

Decline of Nation State

Urbanization

Emerging Economies


Data Does Not Tell a Story

12

Data by itself takes no position and holds no bias.

Biases & other issues only occur when we interpret it, look for predictions, use it to make decisions,.


The Many Perils, Traps and Biases

13

AssumptionsPredictions based on proxiesNot questioning The future = the past

PrivacyBlack boxesUsing data without candidates knowledgeLack of guidelines

BasicsNo clear problem statementPredicting what has no impact

StructuralLack of clean dataSample size too smallToo simplistic/too complex models

BiasesUnintentional/InherentFavoring one set of data over another


What do you want to predict?What is the problem you want to solve?Do you have the right data?Do we have enough data?Do you have enough relevant data?How do we prevent diversity issues?What is a quality hire?How do we define effective?What’s in the algorithm?

Weapons of Conformity & Discrimination?

14

Only 14% of organizations have data to prove the positive business impact of their assessment strategy. -Aberdeen 2014


AssumptionsWhat are we assuming as we search?How valid are our search criteria? How do you know?

Scoring algorithmsWhat is in the algorithm?How do we know it is actually measuring what we think it is?Is it discriminating?Is it fostering clones?How do we introduce diverse thinking?

Sourcing

15


Is a Facebook/LinkedIn profile accurate?Can they predict ability, skill, or job performance?

Using Facebook or LinkedIn

16

“. . .researchers hired HR types to rate hundreds of college students’ Facebook pages according to how employable they seemed.

. . .the over 500 guinea pigs, just 56 of the employers responded. So the sample is small, but the researchers found a strong correlation between those employers’ reviews and the employability predictions they had made based on folks’ profile pages.”


Check out a candidate’s social media profiles – even informallyUse any of the social media info to influence your opinion of a candidateDo not let a candidate know if you lookedDo not let them know what you looked for

Social Media & Privacy

17

Potential legal problems if you. . .


What are the criteria?Does personality testing correlate with performance?Do we want everyone the same?

Assessment

18

The personality test is a black box, and it’s not clear what it is actually assessing, and whether using it constitutes discriminatory hiring practices.


What are the criteria?What is the context?When and to who was the comment made?

Sentiment Analysis

19

“Even in the best of circumstances, [sentiment analysis] is only 65% to 70% accurate.”

-Susan Etlinger, analyst Altimeter Group


Matching algorithmsAccuracy? Relevance? Privacy? Discrimination?

Passive AssessmentPrivacy? Relevance? Accuracy?

ChatbotsAssumptions? Discrimination?

Online Assessments/gamesPrivacy? Relevance? Correlation=Causation?

A Few Emerging Tools

20

How transparent are the vendors about their algorithms and assumptions?


Could We Automate Selection & Assessment? Should We?

21

People are complex, contradictory, and so varied that even complex tools and algorithms are rarely accurate. We will continue to need human judgement and tolerance.


Most of the tools we are using today create as many questions as answers.No single test can predict anything with high certainty.Many tools use proxies, which may not represent reality.We tend to validate a tool when it confirms our suspicions.Sample sizes are often way too tiny to be valid.There is inherent bias in almost all screening.People are highly complex and there are no simple ways to say one person is better than another.

Unfortunately. . .

22


PA can provide insight and validate or disprove assumptions.Can augment human judgement.Valuable when used responsibly following openness guidelines. Can provide early warning that employees are unhappy or are thinking about leaving.Can identify competencies and skills and predict their value to a particular role.

Predictive Analytics

23


Have a privacy disclosure policy that is shared with candidates and is on your career site.Let candidates know why you have rejected them, especially if based on social media information.Have clear definitions of what you are looking for and how you will know when you find it.Train recruiters in what is acceptable, reliable, and accurate.Always use human judgement along with any AI or other assessment or prediction.

Good Predictive Analytics Practices

24


. . .use more than one test or predictive tool.

. . .when choosing a tool, know what is in the algorithm. Know what is being looked for, tested, analyzed, and how each factor is weighted.. . .make sure proxies are valid and really predictive.. . .not adopt tools hastily and without careful thought

and knowledge.. . .always question our own assumptions and beliefs.

We should. . .

25

© 2016, Future of Talent Institute26

THANKS

YOUR THOUGHTS & QUESTIONS

Follow me on Twitter: @kwheelerEmail: [email protected]

promise and perils of predictive analytics

Business