how much faith should we have in data? - odi friday lunchtime lecture
TRANSCRIPT
Jeni [email protected]
Decisions, decisions…
How to travel around London
Where to send my kids to school
https://www.flickr.com/photos/wikidave/4186909251
How much to pay my employees
https://www.flickr.com/photos/59937401@N07/5929622407
https://www.flickr.com/photos/141776778@N02/27323692585
Which way to vote
Humans are bad at decision making
AnchoringAvailabilitySubstitutionOptimism & loss aversionFramingSunk cost
Data to the rescue!
https://www.flickr.com/photos/pkwflickr/6188760566/in/album-72157627764211574/
Roads help us navigate to a location.Data helps us navigate to a decision.
How to travel around London
Where to send my kids to school
How much to pay my employees
Which way to vote
Data can definitely help usmake better decisions
So we are starting todrive decisions with data
What adverts we see
How we are treated when we're ill
Where and how we drive
https://www.flickr.com/photos/44124348109@N01/4044601053
But data-based decisionsare not necessarily good
How our photos get tagged
How much we get fined
http://www.theregister.co.uk/2017/02/08/ministry_justice_conviction_by_computer_scheme_go_ahead/
https://www.theguardian.com/australia-news/2016/dec/29/government-confident-in-centrelink-debt-compliance-system-despite-reported-errors
Whether we get issued a debt notice
https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
How long we spend in prison
Code has bugs
Data has flaws
Biased data collectionErrors in sensor readingsPeople lying or lendingOut of dateFragmentary
That doesn't meanwe shouldn't use dataJust not blindly
Decisions that affect people
What should we do?
1. Be open to scrutiny
https://www.flickr.com/photos/opensourceway/5535034664/
Proactive monitoring
2. Allow appeals and challenges
https://www.flickr.com/photos/sidelong/5923745788
Communication about data use
https://newdigitalrights.projectsbyif.com/data-access-tracker
3. Tailor algorithms to the impact of errors
https://www.flickr.com/photos/benstephenson/27298596
Owe money?Ge
t a d
ebt n
otice
? YesNo
No
Yes
type II errorfalse
negativetype I error
false positive
4. Enable reflection & correction
https://www.flickr.com/photos/phild41/6209957186
Automatically removing bias
Automatically monitoring emergent bias
The Jury by John Morgan
5. Introduce diversity into decision making
Voting?
http://www.flickr.com/photos/83823904@N00/64156219/
Argument?
https://www.flickr.com/photos/jamescridland/4984060658
https://www.flickr.com/photos/quinnanya/1910544811
Human judgement
Be open to scrutinyAllow appeals & challengesTailor algorithms to the impact of errorsEnable reflection & correctionIntroduce diversity into decision making
Decisions should beinformed by data not driven by data
Jeni Tennison – CEO – @JeniT – [email protected]