![Page 1: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/1.jpg)
Reflections
On the meaningful understanding of the logic of automated decision-making
Eric Horvitz
Berkeley Center for Law & TechnologyPrivacy Law Forum: Silicon ValleyMarch 24, 2017
[email protected]://erichorvitz.com
![Page 2: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/2.jpg)
General Data Protection Regulation (EU) 2016/679, 25 May 2018
Article 15 (1)(h):
"existence of automated decision-making…and …meaningful information about the logic involved
Many questions about interpreting Article.
![Page 3: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/3.jpg)
Multiple aspects of AI decision-making pipeline.
![Page 4: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/4.jpg)
Predictions
Decision model
Decisions
Predictive modelML algorithm
Training cases
Dataset
Data Predictions Decisions
![Page 5: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/5.jpg)
Predictions
Data Predictions Decisions
Decision model
Decisions
Predictive modelML algorithm
Training cases
Dataset
![Page 6: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/6.jpg)
AlgorithmTraining
DataPredictions Decisions Use
PersonalData
Logic of processing
meaningful information about the logic involved
![Page 7: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/7.jpg)
AlgorithmTraining
DataPredictions Decisions Use
What, who, when?Accuracy?
Personal Data
Procedure used? Validation?Accuracy?
Decision policy?Values, cost/benefit?
Fully automated?Decision support?
Logic of processing
What, when?
![Page 8: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/8.jpg)
AlgorithmTraining
DataPredictions Decisions Use
Personal Data
TrainingDomain
Localization & Personalization
Validation?Accuracy?
![Page 9: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/9.jpg)
![Page 10: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/10.jpg)
Machine learning “contact lens” for children
March 2017
Localization & Personalization
A. Howard, C. Zhang, E. Horvitz (2010). Addressing Bias in Machine Learning Algorithms: A Pilot Study on Emotion Recognition for Intelligent Systems. IEEE Workshop on Advanced Robotics and its Social Impacts.
![Page 11: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/11.jpg)
Machine learning “contact lens” for children
March 2017
Localization & Personalization
A. Howard, C. Zhang, E. Horvitz (2010). Addressing Bias in Machine Learning Algorithms: A Pilot Study on Emotion Recognition for Intelligent Systems. IEEE Workshop on Advanced Robotics and its Social Impacts.
![Page 12: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/12.jpg)
Moving into High-Stakes Arenas
![Page 13: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/13.jpg)
Readmissions Manager
Localization & Personalization
M. Bayati, M. Braverman, M. Gillam, K.M. Mack, G. Ruiz, M.S. Smith, E. Horvitz (2014). Data-Driven Decisions for Reducing Readmissions for Heart Failure: General Methodology and Case Study. PLOS One Medicine.
![Page 14: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/14.jpg)
Site A Site B
Site C
Site-specific evidence
Site-specific pts, prevalencies
Site covariate dependencies
![Page 15: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/15.jpg)
Site-specific evidence
Site-specific pts, prevalencies
Site covariate dependencies
Site A Site B
Site C
Hospital ACommunity hospital: 180 beds, 10,000 admissions/year.
Hospital BAcute care teaching hospital: 250 beds, 15, 000 inpatients/year.
Hospital CMajor teaching & research hospital: 900 beds, 40,000 inpatient/year.
Same city
![Page 16: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/16.jpg)
AlgorithmTraining
DataPredictions Decisions Use
Personal Data
TrainingDomain
Validation?Accuracy?
LocalTrainingTransferLearning
Challenges of Localization & Personalization
J. Wiens, J. Guttag, and E. Horvitz (2014). A Study in Transfer Learning: Leveraging Data from Multiple Hospitals to Enhance Hospital-Specific Predictions, Journal of the American Medical Informatics Association.
![Page 17: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/17.jpg)
AlgorithmTraining
DataPredictions Decisions Use
Personal Data
TrainingDomain
On Decisions & Values
Validation?Accuracy?
Decisions
![Page 18: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/18.jpg)
AlgorithmTraining
DataPredictions Decisions Use
Personal Data
TrainingDomain
On Decisions & Values
Validation?Accuracy?
Decisions
![Page 19: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/19.jpg)
AlgorithmTraining
DataPredictions Decisions Use
Personal Data
TrainingDomain
On Decisions & Values
Validation?Accuracy?
Decisions
![Page 20: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/20.jpg)
AlgorithmTraining
DataPredictions Decisions Use
Personal Data
TrainingDomain
On Decisions & Values
Validation?Accuracy?
Decisions
![Page 21: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/21.jpg)
AlgorithmTraining
DataPredictions Decisions Use
Personal Data
Narrowing Focus
meaningful information about the logic involved
Interpretability & explainability
![Page 22: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/22.jpg)
M. Zeiler, R. Fergus Visualizing and Understanding Convolutional Networks, Arxiv (2013)
Meaningful Information about ML Procedure?
![Page 23: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/23.jpg)
![Page 24: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/24.jpg)
Rich & open-area of research
One approach: Enable end users to understand
contribution of individual features
What influence does changing observations x have if
other values are not changed?
Interpretability & Explanation of Machine Learning
![Page 25: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/25.jpg)
Interpretability-Power Tradeoff
Logistic regression
Promising space
Neural networks
![Page 26: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/26.jpg)
Interpretability & Explanation
![Page 27: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/27.jpg)
Insights—and Errors
AI Journal 1996
![Page 28: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/28.jpg)
Cooper, et al. 1996
Inspection & Troubleshooting
![Page 29: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/29.jpg)
Inspection & Troubleshooting
(!)
![Page 30: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/30.jpg)
Caruana, et al. 2015
Having our Cake and…
![Page 31: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/31.jpg)
Directions
• Research on understandability & explanation
• Best practices, norms, standards on comprehension
• What aspects of AI pipelines?
• How much detail is “meaningful” understanding?
• For whom? when? What uses & contexts?
![Page 32: Reflections · 2018. 1. 12. · Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site A Site B Site C Hospital A Community hospital: 180 beds, 10,000](https://reader036.vdocuments.site/reader036/viewer/2022071022/5fd6e1293bb5aa01df29bf11/html5/thumbnails/32.jpg)