reinventing fraud prevention & underwriting with machine ... · propriety and confidential...

Propriety and Confidential

Lendit April 2016

Reinventing Fraud Prevention & Underwriting with Machine Learning

Ido Lustig – VP Risk


BlueVine – flexible business lines of credit and invoice factoring

Tel Aviv, Israel• R&D, Risk• 32 Employees

Palo Alto, CA• Biz, Ops, Sales• 34 Employees

$64M in equity and debt financing to date

06/2014 12/201408/2013 03/2014 12/2015

Founded + Seed

Beta launch Full launch Series B Series C


3 QUESTIONSunderlie our underwriting process


?


We need to ask the right questions and answer them like (smart) humans would have.

Machine-human interaction is the key for scale and accuracy


Machine-learning capabilities continually advancing

http://www.bloomberg.com/news/articles/2016-01-03/after-winning-at-chess-this-computer-may-help-decide-on-loans


But it’s still not perfect


- First and last name correlation with loss

- Number of letters in each word

- Total number of letters

- Number of times each letter appears

- Order of letters

- …..


Problem #1 - overfitting

Observations

https://shapeofdata.wordpress.com/2013/03/26/general-regression-and-over-fitting/


Problem #2 – Equal Credit Opportunity Act: things you can’t use (and end up using…)

https://www.washingtonpost.com/news/wonk/wp/2015/05/26/what-your-name-says-about-your-age-state-job-and-political-leanings/


http://fivethirtyeight.com/features/how-to-tell-someones-age-when-all-you-know-is-her-name/




So what’s OK to ask?And what would we be better off not asking at all?


Problem #3 – understanding the outcome

• Clear rejection reasoning (ECOA)

• Debrief and improve your policies


Our 2¢

Insight driven and data backed automation process


Map Questions

Automate Answers

Expose to analysts

Get feedback

Fine tune features

Retrain models


Are documents doctored?

Is the person who she claims

she is? Does the business exist?

Any evidence of criminal activity?

Does the business have a decent

website?

Ask the right questions (fraud example)

Does the activity match the client’s

industry?


Does the business have a decent website? (automation)

Guess URL

✓ Use user provided (www.idosbiz.com) ✓ Use email domain ([email protected]) ✓ Use search engine API (search ido (AND

biz OR business)

Crawl Website

✓ Download website ✓ Classify using internal model ✓ Use Industry as a standard

Website score

➢ Down ➢ Not found ➢ Weak ➢ Medium ➢ High


http://glo4led.com/


http://www.valleyisleaquatics.com/


http://www.royalgranitesandgems.com/

Does the business have a decent website?Does the business have a decent website given the industry?


• Not self-derived from the data • Answer critical questions • Fine tuned, highly accurate • High coverage


Map Questions

Automate Answers

Expose to analysts

Get feedback

Fine tune features

Retrain models


• Hold both calculated and analyst values

• Auto-retrain low performing variables


Map Questions

Automate Answers

Expose to analysts

Get feedback

Fine tune features

Retrain models


• Same process for features, models, and decisions

• For high level models – use tagging (fully automated)


Feature

Deploy

Human Feedb

ack

Auto Retrain

Feature

Deploy

Reality Feedb

ack

Auto Retrain


Automated Decision Rate and Accuracy

0%

25%

50%

75%

100%

Q2 2015 Q3 2015 Q4 2015 Q1 2016 Q2 2016

Coverage Accuracy


Thank You

reinventing fraud prevention & underwriting with machine ... · propriety and confidential...

Documents