methodological problems in cognitive psychologydavid danks institute for human & machine...
TRANSCRIPT
![Page 1: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/1.jpg)
Methodological Problems in Cognitive Psychology
David DanksInstitute for Human & Machine CognitionJanuary 10, 2003
![Page 2: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/2.jpg)
(At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych
1. Novel theoretical possibilities
2. More detailed specification of the experiments being performed
3. Novel experimental designs
![Page 3: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/3.jpg)
Novel Theories
Causal learning community has focused on simple parameter estimation
Given a series of cases, people must rate the “causal strength”Leads to relatively simple psychological theoriesThere are often multiple plausible theories that can fit a particular dataset
![Page 4: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/4.jpg)
Novel Theories
More specifically, causal learning is just parameter estimation for the graph:
Absence of causation is just αI = 0
C1
E
Cn …
α1 αn
![Page 5: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/5.jpg)
Novel Theories
Using Bayes nets suggests novel theories that have not previously been considered in the causal learning community
Specifically, consider the possibility that people are also learning structure, not just parameters
This shift leads to theories such as:Bayesian updating over structures (Tenenbaum)Using interventions to learn structure (Gopnik)
![Page 6: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/6.jpg)
Clarifying Experiments
Typical causal learning experiments just specify the probability distribution over the causes and effect
This formulation can obscure the causal structure to be learned
![Page 7: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/7.jpg)
Clarifying Experiments
Instead, we can use a Bayes net to specify the causal structure (and then make sure the probability distribution fits the graph)
Advantages:1. Can easily determine whether two
experiments are qualitatively different2. Can better determine the range of situations
tested in experiments
![Page 8: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/8.jpg)
Clarifying Experiments
For example, we find that almost all two-cause experiments used one of these two causal structures:
C1 C2
E
C1 C2
E
![Page 9: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/9.jpg)
Novel Experiments
Typical experimental design: Pre-define the causes and effect to fit the probability distributionProvide either a series (or summary) of observations of cases with values for the (earlier defined) causes/effectAsk people to provide a rating of “causal strength”Provide no feedback to the subject until the end of the experiment
![Page 10: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/10.jpg)
Novel Experiments
Using Bayes nets suggests changes to these features of the experimental design:
“Pre-define the causes and effect to fit the probability distribution”Determine the influence of these definitions by using the same probability distribution with different tags for the variables (Waldmann)
![Page 11: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/11.jpg)
Novel Experiments
Using Bayes nets suggests changes to these features of the experimental design:
“Provide either a series (or summary) of observations of cases with values for the (earlier defined) causes/effect”Allow the subject (or the experimenter) to intervene on the variables, rather than simply watching them (Gopnik)
![Page 12: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/12.jpg)
Novel Experiments
Using Bayes nets suggests changes to these features of the experimental design:
“Ask people to provide a rating of ‘causal strength’”Have people choose among graphs that represent causal structure (Steyvers); orHave people make a forced choice about whether “X causes Y ” (Danks)
![Page 13: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/13.jpg)
Novel Experiments
Using Bayes nets suggests changes to these features of the experimental design:
“Provide no feedback to the subject until the end of the experiment”Allow the subjects to see the results of their interventions (Gopnik)
![Page 14: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/14.jpg)
Other Applications?
Use of regression (e.g., in social psychology)Use of latent variable analyses (e.g., PCA)
Bayes net search methods are reliable in more contexts than these two methods – how should they be used in practice?
Application of psychological researchCan we use Bayes nets to model the effects of policies (social and personal) based on psychological research?
![Page 15: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/15.jpg)
Other Applications?
Decision theory that incorporates causationIn the standard theory, decisions are made without using causal structure. Actions look like interventions – can Bayes nets inform theories and experiments of decision-making?
Theory/belief change in childrenCan this process be modeled using Bayes nets? Are novel experiments suggested by that framework?
![Page 16: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/16.jpg)
![Page 17: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/17.jpg)
Constraint-Based & Bayesian Learning
Bayesian learning – it’s what Chris talked about this morning
Establish a probability distribution over graphs (& a distribution over parameters for each graph)Update the distributions based on the observed data using Bayes’ TheoremIn practice, usually approximated using some type of greedy search algorithm
![Page 18: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/18.jpg)
Constraint-Based & Bayesian Learning
Constraint-based learning:Examine one’s data to determine the independencies and associations in that dataDetermine the set of graphs that could possibly have produced data with that pattern
![Page 19: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/19.jpg)
Constraint-Based & Bayesian Learning
Example of constraint-based learning:
X Y Z
X Y Z
X Y Z
X Y,ZY ZX Z | Y
![Page 20: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/20.jpg)
Intervention vs. Observation
We all agree that causal information is useful because it enables us to predict the outcomes of our interventionsSo for Bayes nets to be good models of causation, they must be good models of the effects of interventionsHow do they do that?
![Page 21: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/21.jpg)
Intervention vs. Observation
To model an intervention in a Bayes net, you simply remove all edges into the “intervened-upon” variable, and leave all other edges intactThus, to determine whether an intervention on X changes Y, you just change the graph according to the above rule, and then see whether there is a path from X to Y(note: not exactly right, but close enough)
![Page 22: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/22.jpg)
Intervention vs. Observation
Consider the simplest case of causation: Light Switch Light BulbAn intervention on Light Switch doesn’t change the graph at all (since there are no edges into Light Switch), and so that intervention will matter for Light BulbOn the other hand, intervening on Light Bulbdoes change the graph (removing the edge), and so the intervention doesn’t matter for LS
![Page 23: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/23.jpg)
Intervention vs. Observation
A large (abstract) causal structure:
A B
C
DE
F
![Page 24: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/24.jpg)
Intervention vs. Observation
A large (abstract) causal structure:
Intervene on D!
D
A B
C
E
F
![Page 25: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/25.jpg)
Intervention vs. Observation
A large (abstract) causal structure:
Intervene on D!
D
A B
C
E
F
![Page 26: Methodological Problems in Cognitive PsychologyDavid Danks Institute for Human & Machine Cognition January 10, 2003 (At Least) Three Ways Bayes Nets Can Matter for Cognitive Psych](https://reader035.vdocuments.site/reader035/viewer/2022071401/60ec31cc368476503a198a01/html5/thumbnails/26.jpg)
Intervention vs. Observation
A large (abstract) causal structure:
The intervention on D affects only F!
Intervene on D!
D
A B
C
E
F