![Page 1: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/1.jpg)
Focus on discriminationq Discrimination is a specific type of unfairnessq Well-studied in social sciences
q Political scienceq Moral philosophyq Economicsq Law
q Majority of countries have anti-discrimination lawsq Discrimination recognized in several international human rights laws
q But, less-studied from a computational perspective
![Page 2: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/2.jpg)
What is a computational perspective?Why is it needed?
![Page 3: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/3.jpg)
Case study: Recidivism risk predictionq COMPAS recidivism prediction tool
q Built by a commercial company, Northpointe, Inc.
q Estimates likelihood of criminals re-offending in futureq Inputs: Based on a long questionnaireq Outputs: Used across US by judges and parole officers
q Trained over big historical recidivism data across US q Excluding sensitive feature info like gender and race
![Page 4: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/4.jpg)
COMPAS Goal: Criminal justice reformq Many studies show racial biases in human judgments
q Idea: Nudge subjective human decision makers with objective algorithmic predictionsq Algorithms have no pre-existing biasesq They simply process information in a consistent manner
q Learn to make accurate predictions without race info.q Blacks & whites with same features get same outcomesq No disparate treatment & so non-discriminatory!
![Page 5: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/5.jpg)
Is COMPAS fair to all groups?Black Defendants
High Risk Low RiskRecidivated 1369 532
Stayed Clean 805 990
White DefendantsHigh Risk Low Risk
505 461349 1139
![Page 6: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/6.jpg)
Is COMPAS fair to all groups?Black Defendants
High Risk Low RiskRecidivated 1369 532
Stayed Clean 805 990
White DefendantsHigh Risk Low Risk
505 461349 1139
False Discovery Rate: 805 / (805 + 1369) = 0.37 349 / (349 + 505) = 0.40
![Page 7: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/7.jpg)
Is COMPAS fair to all groups?Black Defendants
High Risk Low RiskRecidivated 1369 532
Stayed Clean 805 990
White DefendantsHigh Risk Low Risk
505 461349 1139
False Discovery Rate: 805 / (805 + 1369) = 0.37 349 / (349 + 505) = 0.40
False Omission Rate: 532 / (532 + 990) = 0.35 461 / (461 + 1139) = 0.29
![Page 8: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/8.jpg)
Is COMPAS fair to all groups?
q Northpointe: False discovery & omission rates for blacks & whites are comparable
q So YES!
Black DefendantsHigh Risk Low Risk
Recidivated 1369 532Stayed Clean 805 990
White DefendantsHigh Risk Low Risk
505 461349 1139
False Discovery Rate: 805 / (805 + 1369) = 0.37 349 / (349 + 505) = 0.40
False Omission Rate: 532 / (532 + 990) = 0.35 461 / (461 + 1139) = 0.29
![Page 9: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/9.jpg)
Is COMPAS non-discriminatory?Black Defendants
High Risk Low RiskRecidivated 1369 532
Stayed Clean 805 990
White DefendantsHigh Risk Low Risk
505 461349 1139
![Page 10: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/10.jpg)
Is COMPAS non-discriminatory?Black Defendants
High Risk Low RiskRecidivated 1369 532
Stayed Clean 805 990
White DefendantsHigh Risk Low Risk
505 461349 1139
False Positive Rate: 805 / (805 + 990) = 0.45 349 / (349 + 1139) = 0.23
![Page 11: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/11.jpg)
Is COMPAS non-discriminatory?Black Defendants
High Risk Low RiskRecidivated 1369 532
Stayed Clean 805 990
White DefendantsHigh Risk Low Risk
505 461349 1139
False Positive Rate: 805 / (805 + 990) = 0.45 349 / (349 + 1139) = 0.23
False Negative Rate: 532 / (532 + 1369) = 0.29 461 / (461 + 505) = 0.48
![Page 12: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/12.jpg)
Is COMPAS non-discriminatory?
q ProPublica: False positive & negative rates are considerably worse for blacks than whites!q Constitutes discriminatory disparate mistreatment
Black DefendantsHigh Risk Low Risk
Recidivated 1369 532Stayed Clean 805 990
White DefendantsHigh Risk Low Risk
505 461349 1139
False Positive Rate: 805 / (805 + 990) = 0.45 >> 349 / (349 + 1139) = 0.23
False Negative Rate: 532 / (532 + 1369) = 0.29 << 461 / (461 + 505) = 0.48
![Page 13: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/13.jpg)
Why are error comparisons so different?
q Impossibility result: [Kleinberg ’17, Chouldechova ‘17]q When recidivism ratios for blacks & whites differ,
no non-trivial solution can achieve equal FDR, FOR, FPR, FNR!q Can equalize at most two out of the four error rates!
Black DefendantsHigh Risk Low Risk
Recidivated 1369 532Stayed Clean 805 990
White DefendantsHigh Risk Low Risk
505 461349 1139
Recidivism Ratio: (1369 + 532) : (805 + 990) (505 + 461) : (349 + 1139) = 1.06 : 1.00 = 0.65 : 1.00
![Page 14: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/14.jpg)
Why, a computational perspective?q Formal interpretations of discrimination can help us
understand the notions better
q Reveals the inherent trade-offs between multiple measures of discrimination and their utility
q Another example: Fairness of random judge selection q Suppose you have N fair / unfair judges
q They have equal FPR / FNR / FOR / FDR for different racial groupsq Does assigning cases to judges randomly affect fairness?
![Page 15: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/15.jpg)
Computational Interpretations (measures) of Discrimination [WWW ‘17]
![Page 16: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/16.jpg)
Defining discriminationq A first approximate normative / moralized definition:
wrongfully impose a relative disadvantage on persons based on their membership in some salient social group e.g., race or gender
q Challenge: How to operationalize the definition?q How to make it clearly distinguishable, measurable, &
understandable in terms of empirical observations
![Page 17: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/17.jpg)
Need to operationalize 4 fuzzy notions1. What constitutes a relative disadvantage?
2. What constitutes a wrongful imposition?
3. What constitutes based on?
4. What constitutes a salient social group?
![Page 18: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/18.jpg)
Need to operationalize 4 fuzzy notions1. What constitutes a relative disadvantage?
2. What constitutes a wrongful imposition?
3. What constitutes based on?
4. What constitutes a salient social group?q Defined by anti-discrimination laws: Race, Gender
![Page 19: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/19.jpg)
Need to operationalize 4 fuzzy notions1. What constitutes a relative disadvantage?
2. What constitutes a wrongful imposition?
3. What constitutes based on?q Do not use salient group information in training or deploymentq Use during training, but not deploymentq Use during both training and deployment
4. What constitutes a salient social group?
![Page 20: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/20.jpg)
Need to operationalize 4 fuzzy notions1. What constitutes a relative disadvantage?
2. What constitutes a wrongful imposition?
3. What constitutes based on?
4. What constitutes a salient social group?
![Page 21: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/21.jpg)
Operationalizing discrimination
q Consider binary classification using user featuresF1 F2 … Fm Z
User1 x1,1 x1,2 … x1,m Z1
User2 x2,1 x2,m Z2
User3 x3,1 x3,m Z3
… … … …Usern xn,1 xn,2
… xn,m Zn
Decision
AcceptRejectReject
…Accept
Decision outcomes should not be relatively disadvantageous to social (sensitive feature) groups!
![Page 22: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/22.jpg)
Relative disadvantage measure 1: Disparate treatment
Feature 1
Feat
ure
2
Measures the fraction of users whose outcomes change, when their sensitive features are changed
B1
B2
DT (B1, B2)= 15 / 30 = 0.5
![Page 23: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/23.jpg)
Relative disadvantage measure 1: Disparate treatment
Feature 1
Feat
ure
2
Measures the fraction of users whose outcomes change, when their sensitive features are changed
B
DT (B) = 0 / 30 = 0
![Page 24: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/24.jpg)
Measures direct discriminationq Based on counter-factual reasoning
q Most intuitive measure of discrimination
q To achieve parity treatment: Ignore sensitive features, when defining the decision boundary
q Situational testing for discrimination discovery checks for disparate treatment
q More formally:
![Page 25: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/25.jpg)
Relative disadvantage measure 2: Disparate impact
Feature 1
Feat
ure
2
Measures the difference in fraction of positive (negative) outcomes for different sensitive feature groups
B1
DI (B1) = 7/15 - 1/15 = 0.4
![Page 26: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/26.jpg)
Relative disadvantage measure 2: Disparate impact
Feature 1
Feat
ure
2
Measures the difference in fraction of positive (negative) outcomes for different sensitive feature groups
B1
B2
DI (B1) = 0.4DI (B2) = 7/15 - 6/15 = .06
![Page 27: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/27.jpg)
Measures indirect discriminationq Observed in human decision making
q Indirectly discriminate against specific user groups using their correlated non-sensitive attributesq E.g., voter-id laws being passed in US states
q Doctrine of disparate impactq A US law applied in employment & housing practicesq Proportionality tests over decision outcomes
![Page 28: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/28.jpg)
A controversial measureq To achieve parity impact: Select equal fractions of
sensitive feature groupsq More formally:
q In Law:q Critics: There exist scenarios where disproportional
outcomes are justifiableq Supporters: Provision for business necessity exists
q Though the burden of proof is on employers
q In ML: Use, when labels in training data are biased
![Page 29: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/29.jpg)
Relative disadvantage measure 3: Disparate mistreatment
Feature 1
Feat
ure
2
Measures the difference in fraction of accurate outcomes for different sensitive feature groups
B1
DM (B1) = 13/15 - 9/15 = .26
![Page 30: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/30.jpg)
Relative disadvantage measure 3: Disparate mistreatment
Feature 1
Feat
ure
2
Measures the difference in fraction of accurate outcomes for different sensitive feature groups
B1
B2
DM (B1) = 13/15 - 9/15 = .26DM (B2)= 10/15 – 10/15 = 0
![Page 31: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/31.jpg)
Learning disparate mistreatment
![Page 32: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/32.jpg)
Learning disparate mistreatment
q Optimal (most accurate / least loss) linear boundaryq But, how do machines find (compute) it?
q The boundary was computed using
![Page 33: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/33.jpg)
Learning disparate mistreatment
q Optimal (most accurate / least loss) linear boundary
![Page 34: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/34.jpg)
Learning disparate mistreatment
q Optimal (most accurate / least loss) linear boundaryq Makes few errors for yellow, lots of errors for blue!
q Commits disparate mistreatment: ≠
![Page 35: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/35.jpg)
Measures indirect discriminationq In decision making scenarios, where we have unbiased
ground truth outcomes
q To achieve parity mistreatment: Provide accurate outcomes for equal fractions of sensitive feature groups
q More formally:
q The above overall inaccuracy rate measure can be further broken down into its constituent FPR, FNR, FDR, and FOR
![Page 36: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/36.jpg)
Summary: 3 discrimination measures1. Disparate treatment: Intuitive direct discrimination
q To avoid:
2. Disparate impact: Indirect discrimination, when ground-truth may be biased
q To avoid:
1. Disparate mistreatment: Indirect discrimination, whenground-truth is unbiased
q To avoid:
![Page 37: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/37.jpg)
From Parity to Preference-based Discrimination Measures [NIPS ‘17]
![Page 38: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/38.jpg)
Recap: Defining discriminationq A first approximate normative / moralized definition:
wrongfully impose a relative disadvantage on persons based on their membership in some salient social group e.g., race or gender
![Page 39: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/39.jpg)
Recap: Operationalize 4 fuzzy notions1. What constitutes a relative disadvantage?
2. What constitutes a wrongful imposition?
3. What constitutes based on?
4. What constitutes a salient social group?
![Page 40: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/40.jpg)
Need to operationalize 4 fuzzy notions1. What constitutes a relative disadvantage?
2. What constitutes a wrongful imposition?
3. What constitutes based on?
4. What constitutes a salient social group?
![Page 41: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/41.jpg)
Is disparity in group error/acceptance rates wrong in all scenarios?
![Page 42: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/42.jpg)
Parity error rates aren’t pareto-optimal
Feature 1
Feat
ure
2
Parity error rates: Picks non pareto-optimal B2 over B1 Preferred error rates: Picks pareto-optimal B1 over B2
B1B2
Accuracy (B1) = 13/15 15/15
Accuracy (B2) = 09/15 09/15
![Page 43: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/43.jpg)
Measures bargained discriminationq Inspired by bargaining solutions in game-theory
q Disagreement (default) solution is parity!q Both groups try to avoid tragedy of parity
q Selects pareto-optimal boundaries over group accuracies
q More formally:P(ŷ ≠ y | Xz=0, W) ≥ P(ŷ ≠ y | Xz=0, Wparity)P(ŷ ≠ y | Xz=1, W) ≥ P(ŷ ≠ y | Xz=1, Wparity)
![Page 44: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/44.jpg)
Are group-based decision boundaries discriminatory in all scenarios?
![Page 45: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/45.jpg)
Group-based decisions can be envy-free
Feature 1
Feat
ure
2 B1
B2
Impact (B1) = 5/15 0/15
Impact (B2) = 0/15 6/15
Parity treatment: Disallows group-based boundaries B1, B2 Preferred treatment: Allows envy-free boundaries B1, B2
![Page 46: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/46.jpg)
Measures envy-free discriminationq Preferred treatment allows group-conditional boundaries
q Yet, ensure they are envy-freeq No lowering the bar to affirmatively select certain user groups
q Can be defined at individual or group-level
q More formally:P(ŷ = 1 | Xz=0, Wz=0) ≥ P(ŷ = 1 | Xz=0, Wz=1)P(ŷ = 1 | Xz=1, Wz=1) ≥ P(ŷ = 1 | Xz=1, Wz=0)
![Page 47: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/47.jpg)
Summary: From parity to preference-based measures of discrimination
q Refined our three measures of discriminationq Disparate treatment / impact / mistreatmentq Preferred treatment / impact / mistreatment
q The new measures allow group-conditional, envy-free, pareto-optimal boundariesq Can also be combined with one-another and parity measures
![Page 48: Focus on discriminationcourses.mpi-sws.org/hcml-ws18/lectures/Lecture_2.pdf · Focus on discrimination qDiscrimination is a specific type of unfairness qWell-studied in social sciences](https://reader033.vdocuments.site/reader033/viewer/2022050201/5f5546ff6da9c47cc953bd5a/html5/thumbnails/48.jpg)
Operationalizing 4 fuzzy notionsq What constitutes a salient social group?
1. Defined by anti-discrimination laws: Race, Genderq What constitutes based on?
1. Using salient group information in training or deployment2. Using salient group information in deployment, but not training3. Using salient group information in non envy-free boundaries
q What constitutes a relative disadvantage?1. Disparity in outcomes for similar users across groups 2. Disparity in error rates across groups3. Disparity in acceptance rates across groups
q What constitutes a wrongful imposition?1. Any relative disadvantage for any group2. Non pareto-optimal or lower than parity advantage for any group