Comparative Politics

Methodology:Natural Sciences and Political Science

Rein Taagepera Predictive Vs Postdictive models

The major goal of sciences is to explain in a way that can lead to prediction:

“this should be so, because logically ...” instead of

“this is so, an that’s it”

The purely postdictve approach reverses the usual role of scientist and statistician,

and crowds out creative thinking

How predictive and postdictive models works

Features for the construction of a quantitatively predictive model

• to use boundary conditions and other logical constraints to establish anchor points;

• to look for simplest set of equations that does not violate the logical constraints;

• to wonder about the possible range of values of coefficients and constants (some limits are firm other are fluid);

• to use the means of data to estimate some coefficients in the predictive model (still a step that precedes regression)

Most numbers published in political science are dead on arrival

Conclusions

Conceptual building has to be brought back in political science

Political Science has to widen the tool-kit complementing passive postdictive models with creative predictive ones

Stephen Coleman Testing theories with qualitative and quantitative predictions

Physics has developed mainly withtout modern statistical hypotheses testing

Social sciences apply a standard quantitative approach to theory development and testing: multivariate analysis

It is argued that:• Confidence in statistical methods to construct theories is misplaced

• Theory testing is more productive if we combine definitive theory generated predictions with statistical models

Failures in theory testing with observational data

• Statistical models of behaviour are not unique

• Weak power of prediction

• Lack of replication

• False positives

• Method uncertainty

Then what to do?Need to improve statistical methods

• split observations in two groups to reduce false positives: the model is estimated on one and tested on the other (need for enough data)

• run robust regression methods to moderate the impact of extreme values (do not solve model uncertainty)

• use generalised linear models (not a panacea)

Critiques moved to econometricsSummers:

• Fails to find causal relationships not obvious from the start• Replication is absent and rarely important• Methodological advances are confused with substantive progress• Problems in statistical testingPesaran and Smiths:

• no agreement on method of judging whether the conditional predictions match the data (what should reject? data, method, theory)

Dharmapala and McAleer:

• inability to test questions the practicality of the Popper’s theory falsification as the path to scientific progress

Psychology of theory constructioncan help Soc. Sc.

Coleman suggests to apply the method of psychology construction to social sciences:

try to explain a phenomenon by starting with one ore more specific theories that would explain it; then derive and test a prediction that has a high probability of being associated with the prospective theory but otherwise is very unlikely to be observed.

Examples of predictive theory testing in natural sciences

• Newton’s prediction of elliptical orbits of the planets from the inverse square law of gravitation;

• various experiments confirming the wave theory of light;

• Maxwell’s prediction of electromagnetic waves from a mathematical model;

• the Michelson-Morley experiment that disproved the existence of the ether and confirmed the constant velocity of light;

• Kelvin’s prediction of absolute zero temperature;

• ...

Typology of predictions

• prediction of a constant or invariant (like the speed of light or a freezing point)

• prediction of a specific number;

• prediction of a symmetry, often derived from a mathematical model;

• prediction of a topological fixed point;

• prediction of a limit or constraint, dynamic limited cycle

• prediction of a specific spatial pattern

• prediction of a statistical distribution (possibly unusual)

• prediction that data will have a “signature” - a unique mathematical shape

• sometimes a predicted result is better tested or more evident when the data are transformed and analysed in a different coordinate system

Conclusions

• Careful about drawing theoretical inferences from observational data;

• A better approach is to develop definitive testable predictions that will clearly reveal which of several possible explanations is most likely correct;

• Statistical analysis should then be used to build confidence in the results

Josep Colomer What other sciences look like?

• mathematical properties of natural sciences equations;• traditional objections to extending natural science model to economics;• features of math. equations to be shared in political science.

Properties of mathematical formulae in physics equations

• few variables, and even fewer constants;

• all the variables involved (time, space, matter,...) are well defined and can be measured with well established instruments;

• few are sums or subtractions, mostly are multiplications, divisions and powers;

• almost half of them follow the same pattern

y = aΠxbi : any variable can be obtained by multiplication/division of one or more other variables times a constant

Properties of chemistry’s equations

• variables are well identified elements;

• equations show the quantities of each component, which can be precisely measured with usual instruments;

• every chemical compound, formed by one or more element has a formula that cannot be altered;

• every equations makes a precise prediction, which can be effectively tested;

• mathematical signs are weak, but directionality is specified

Objections to the use of Nat. Sc. model in Economics (econometrics)

• range of validity of economic theorems: human/social world changes more than the natural one imposing more constraining territorial and temporal limits on the validity of hypotheses and postulates;

• the influence of the observer on the subject being studied has a greater degree of influence in social sciences.

Consequences:

• directionality should be specified in any model;• decisions made by actors’ given specific constraints and incentives may be specified in order to account for expected outcome

Conclusions:lessons & warnings towards a

polimetrcs

• a simple and relevant equation should include a small number of well-defined variables;

• parameters and coefficients should be measured and used;

• equations based on reasonable hypotheses about the relationship among variables;

• non-linear forms of equation (multplication, divisions, powers, derivatives);

• directionality of the relationship should be specified;

• hypotheses should be postulated within explicit territorial and temporal limits;

• actors’ decision regarding the manipulation of some variables should occupy a central role in the analysis.