how much faith should we have in data? - odi friday lunchtime lecture

Jeni Tennison@JeniTtheODI.org

Decisions, decisions…

How to travel around London

Where to send my kids to school

https://www.flickr.com/photos/wikidave/4186909251

How much to pay my employees

https://www.flickr.com/photos/59937401@N07/5929622407

https://www.flickr.com/photos/141776778@N02/27323692585

Which way to vote

Humans are bad at decision making

AnchoringAvailabilitySubstitutionOptimism & loss aversionFramingSunk cost

Data to the rescue!

https://www.flickr.com/photos/pkwflickr/6188760566/in/album-72157627764211574/

Roads help us navigate to a location.Data helps us navigate to a decision.

How to travel around London

Where to send my kids to school

How much to pay my employees

Which way to vote

Data can definitely help usmake better decisions

So we are starting todrive decisions with data

What adverts we see

How we are treated when we're ill

Where and how we drive

https://www.flickr.com/photos/44124348109@N01/4044601053

But data-based decisionsare not necessarily good

How our photos get tagged

How much we get fined

http://www.theregister.co.uk/2017/02/08/ministry_justice_conviction_by_computer_scheme_go_ahead/

https://www.theguardian.com/australia-news/2016/dec/29/government-confident-in-centrelink-debt-compliance-system-despite-reported-errors

Whether we get issued a debt notice

https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

How long we spend in prison

Code has bugs

Data has flaws

Biased data collectionErrors in sensor readingsPeople lying or lendingOut of dateFragmentary

That doesn't meanwe shouldn't use dataJust not blindly

Decisions that affect people

What should we do?

1. Be open to scrutiny

https://www.flickr.com/photos/opensourceway/5535034664/

Proactive monitoring

2. Allow appeals and challenges

https://www.flickr.com/photos/sidelong/5923745788

Communication about data use

https://newdigitalrights.projectsbyif.com/data-access-tracker

3. Tailor algorithms to the impact of errors

https://www.flickr.com/photos/benstephenson/27298596

Owe money?Ge

t a d

ebt n

otice

? YesNo

No

Yes

type II errorfalse

negativetype I error

false positive

4. Enable reflection & correction

https://www.flickr.com/photos/phild41/6209957186

Automatically removing bias

Automatically monitoring emergent bias

The Jury by John Morgan

5. Introduce diversity into decision making

Voting?

http://www.flickr.com/photos/83823904@N00/64156219/

Argument?

https://www.flickr.com/photos/jamescridland/4984060658

https://www.flickr.com/photos/quinnanya/1910544811

Human judgement

Be open to scrutinyAllow appeals & challengesTailor algorithms to the impact of errorsEnable reflection & correctionIntroduce diversity into decision making

Decisions should beinformed by data not driven by data

Jeni Tennison – CEO – @JeniT – jeni@theodi.org