the metaethics of training data · 2020. 7. 15. · the metaethics of training data christoph...

The Metaethics of Training Data

Christoph Merdes

ZiWiS (FAU)

2. July 2019

Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 1 / 22

1 Introduction (and Motivation)

2 Labeling the Data

3 The Problem of Representation

4 Conclusion and Outlook


Plan


2 Labeling the Data




The Task

Autonomy of robotic agents increases (Reed and Jones, 2013;Anderson et al., 2006)More situations require morally relevant decision to be made by therobotMany of those decisions cannot be foreseen in detail

Task: Given some morally relevant action, how can a robotic agent decidewhether or too which degree it is morally good?


Bottom-Up

Problems:1 Difficult to fully specify all possible scenarios in advance (framing

problem)2 No actual agreement about explicit theory of normative ethics to be

implemented

Bottom-Up AgentsArtificial agents could, similar to humans, acquire moral judgmentcapabilities by implicit processes, in particular inductive learning andartificial selection (Wallach et al., 2008).

Specifically, Conitzer et al. (2017) suggest machine learning as anapproach to this method of solving our task.


Why Metaethics?

1 Metaethics defines (or reconstructs, explicates, . . . ) the concepts ofthe application domain.

2 Committing to metaethical (and ethical) theories can help us toattack the problem, e.g. by defining a relevant population to samplefrom.

3 In implementing moral machines, we might implicitly commit to(meta)ethical theories, which we should be aware of (e.g. to notclaim more generality than warranted).


Plan


2 Labeling the Data




Ground Truth

ProblemIn ML, it is assumed that there is a ground truth, a fact of the matter thatthe ML system learns to identify or classify reliably as what it is. Moraltruth?

Ontological problem: It is controversial whether there is any moraltruth, if so, if it is independent from rational judgment capability.Epistemic problem: Even if there is a ground truth, it is stillcontroversial by which means it is epistemically accessible.

In a supervised learning context, our metaethical stance has to inform ourdata collection and interpretation of results!


Intuition

Assume, for the time being, that there is a ground truth. How would weknow about it?

Moral IntuitionHuman agents have moral intuitions (Sosa, 2007), which allow them toimmediately reject or accept moral judgments with high accuracy; this canbe understood similar, but not quite the same as perceptual capabilities.

Problems: Missing error theory and high level of disagreement


Expert Intuition

IdeaEven though we have no explicit error theory, we can assume that agentswho deliberate ethical problems a lot will have an increased competency,hence allowing us to infer reliable epistemic access to judgments.

Minor worries: Cultural homogeneity of academic ethicists, remainingdisagreement. . .


Experimental Evidence

Irrational IntuitionExperiments have shown that plausibly intuitive expert judgments onmoral dilemmas are often subject to the same fallacies of reasoning as theaverage person, e.g. order effects (Schwitzgebel and Cushman, 2012).

If this line of research is followed more closely, it might help toactually establish a robust error theory of intuition.For now, it merely shows that expert intuition is subject to commonhuman fallacies.


Common Access

Moral ReasonsAnother worry, different in kind, is that privileging the intuition of apopulation on the basis of a mere indicator (academic education in ethics,frequent deliberation) cannot form the basis of an acceptable theory ofnormative ethics.

Why?

Opaque processes do not provide the right kind of reason to acceptjudgmentMoral agency cannot be imposed from the outside, but it has todevelop autonomously (Kant, 2011)


Common Access

Moral ReasonsAnother worry, different in kind, is that privileging the intuition of apopulation on the basis of a mere indicator (academic education in ethics,frequent deliberation) cannot form the basis of an acceptable theory ofnormative ethics.

Why?Opaque processes do not provide the right kind of reason to acceptjudgmentMoral agency cannot be imposed from the outside, but it has todevelop autonomously (Kant, 2011)


Well-Considered Judgments

Deliberate JudgmentInstead of using intuition, we should employ carefully deliberatedjudgments as the input to moral ML.

Which population should they come from?How to ensure that they are honest and well deliberated? (Cost!)What additional information is actually fed into the process bymoving from intuitions to considered judgments?

But: This seems to be workable as a starting point; the ML algorithm isthen understood to learn the implicit commitments and moral theory ofthe judging agents, not what is right/good itself.


Plan


2 Labeling the Data




Representation

ProblemFor moral learning, there is no obvious representation; moral dilemmas, forexample, are abstract entities that can be represented in various ways,with different technical and metaethical implications.

Options:Standardized formal representationNatural languageVisual representation. . . ?


Formal Representation

Conitzer et al. (2017) argue that it is practically necessary to represent themoral problem under consideration formally. They can be variedautomatically to support data collection, they simplify learning and theyallow for more interpretability. Objections:

They assume that game-theoretic models are faithful representationsof moral problems; however, as they realize themselves, key featuresin particular for non-consequentialist theories are not represented well,raising the question of what the alternative is.Related is the issue that a formal representation likely presupposes orat least favors a point of view in normative ethics; but the motivationto work bottom-up was to avoid making such a choice.


Natural Language

IdeaHumans seem to be capable to pass moral judgment on the basis ofnatural language descriptions; natural language can be processed by ML;hence, we could use NL representations.

Difficult to systematically vary for data generationMay contain latent commitments to theoryIf the AMM relies on first extracting features from the input, anytheoretical loading of the feature selection process (e.g. a dictionaryof moral vocabulary (Garten et al., 2016)) are added to the mix.

For these reasons, I would advocate for some standardized NLrepresentation, similar to what is sometimes used in experiments.


Visual Input

AlternativeSome philosophers have argued that we see morally right and wrong as wesee other features of the world; hence, couldn’t we use visualrepresentations of a scenario to judge?

Metaethically dubious; the notion of moral perception is ratherunclear.Hypothesis: Human agents actually will be very unreliable in judginggraphical depictions of actions as right or wrong, as a larger amountof potentially idiosyncratic background assumptions is required as in alinguistic or formal representation.


Plan


2 Labeling the Data




Outlook

Is it possible to understand ML as a constructivist’s ideal procedure?(Implies fully rejecting the idea of a ground truth!)How should we deal with inconsistency in the collected training data?Which theory of normative ethics provides the best fit for the MLbehavior, and can we improve ML by starting explicitly from such atheory? (e.g. theory informed feature pre-selection)What implications are there of actually employing artificial moralagents that have been trained bottom-up on a large scale?


References

Anderson, M., S. L. Anderson, and C. Armen (2006). Medethex: a prototype medical ethicsadvisor. In Proceedings Of The National Conference On Artificial Intelligence, Volume 21,pp. 1759–1765. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999.

Conitzer, V., W. Sinnott-Armstrong, J. S. Borg, Y. Deng, and M. Kramer (2017). Moraldecision making frameworks for artificial intelligence. In Thirty-First AAAI Conference onArtificial Intelligence.

Garten, J., R. Boghrati, J. Hoover, K. M. Johnson, and M. Dehghani (2016). Morality betweenthe lines: Detecting moral sentiment in text. In Proceedings of IJCAI 2016 workshop onComputational Modeling of Attitudes.

Kant, I. (2011). Immanuel Kant: Groundwork of the Metaphysics of Morals: A German–Englishedition. The Cambridge Kant German-English Edition. Cambridge University Press.

Reed, G. S. and N. Jones (2013). Toward modeling and automating ethical decision making:design, implementation, limitations, and responsibilities. Topoi 32(2), 237–250.

Schwitzgebel, E. and F. Cushman (2012). Expertise in moral reasoning? order effects on moraljudgment in professional philosophers and non-philosophers. Mind & Language 27(2),135–153.

Sosa, E. (2007). Experimental philosophy and philosophical intuition. Philosophicalstudies 132(1), 99–107.

Wallach, W., C. Allen, and I. Smit (2008). Machine morality: bottom-up and top-downapproaches for modelling human moral faculties. Ai & Society 22(4), 565–582.


Questions?

Increasing autonomy of machines requires AMMBottom-up offers a way around framing and theoretical disagreementHowever, already the selection of training data raises serious technicaland (meta)ethical questions:

Who can label the data reliably, even if there is a ground truth?What representation should be chosen, which option comes with whatcommitments?


the metaethics of training data · 2020. 7. 15. · the metaethics of training data christoph...

Documents