Aspiring Minds

www.aspiringminds.com

Outcome predictions using test scores

Who can learn programming through a short 3-month training?

Varun Aggarwal

Presented at Education Workshop at ICML, 2015

Case of a large IT Services company

• Companies wants to hire candidates who when trained for 3-months will learn programming to be deployed as software engineers.

• 1371 candidates tested on AMCAT at the time of joining a large IT company. Modules tested:• English Comprehension• Quantitative Ability• Logical Reasoning• Computer Programming

• Company shares who succeeded and who did not. Candidates classified as High Performers, Average Performers and Mid-Performers.

The Model Type• Should be theoretically plausible

• Models need to be interpretable

• Models need to be simple

• Trade-off models between type 1 and type 2 error required

Class 1

Class 0

Not allowed

L > 400E + 0.4*L > 870E + 0.7*Q > 980

English

Logi

cal

Designed a genetic algorithm based system to search the model-space

HP 76.66

AP 15.36

PP 7.98

Test Shortlist rate40%

AP Rejected 26.88%

PP Rejected32.88%

HP 82.49

AP 11.24

PP 6.27

Test Shortlist rate29.31 %

AP Rejected 39.17%

PP Rejected47.26%

HP 86.09

AP 9.1

PP 4.8

Test Shortlist rate21.72%

AP Rejected 50.75%

PP Rejected59.62%

Test shortlist Rate: 31%

Poor Performers11.89%

Average Performer18.38%

High Performers69.73%

CURRENT ORGANIZATION

We can help the organization…

Knowledge Discovery: Quant Ability not a useful parameter

Type 1

Type

2

15 20 25 30 35 40 45 500

10

20

30

40

50

60

ELELC

Shortlist Rate

% lo

w p

erfo

rmer

s rem

oved

COUNTER-INTUITIVE TO BUSINESS PEOPLE!!!

CP model gets you lower type 1 and higher SR

Knowledge Discovery: Exposure to programming reduces need of logical ability

1• Computer > 360

2• 0.47*English + 0.98*Computer + Logical > 1196

• The first filter eliminates bottom 28% candidates in Programming. •Above a minimum benchmark, lack of Programming knowledge can be offset by high Logical Ability and English skills.

1• 0.42* English + Logical > 722

2• Logical > 464

OR

• Logical ability needs to above the mean.• Above a minimum benchmark, lack of logical Ability can be offset by high English comprehension skills

Who gets a job? Is there a bias?Who can succeed in a short-duration programming training?What behaviors lead to consistent job failures?

Learnings