130 Book reviews

report’s coverage is full and additional refer- asked whether various measures such as MAPEences are given. In fact, many of the products ‘‘can be computed’’. Some of the statisticalare given limited coverage because the develop- packages answered a wholehearted ‘‘yes’’, buters have chosen not to contribute, a limitation in this disguises the fact that the user would haveitself if that package is being considered for substantial programming to do!)commercial use. Timeliness is also an issue in The last chapter offers ‘‘personal advice onthat software products apparently have a short how to choose a forecasting package’’. Theshelf-life. But the authors argue that the core advice is always sensible and in my experienceroutines included in the packages have a longer mostly ignored by even large experienced com-life span, a conclusion I would agree with in panies. If I had to select one particular recom-that exponential smoothing, the basis of many mendation it would be ‘‘to set up a carefullyof the routines has been around in much the designed study on the basis of time series whichsame form for more than forty years. typically occur in your company’’, not relying

The results of the comparisons are compli- on generic comparisons (such as the variouscated in that the comparisons attempted are forecasting competitions) or the blandishmentsmultidimensional. The authors summarise the of the sales force with proprietary methodssystems through a subjective scoring system, which they cannot explain. One extra recom-graphing key aspects of the results. Examples of mendation I would add in however – where youthe dimensions considered include the type of believe accuracy matters, concentrate on the keysystem, whether the package is primarily suited dimension of choosing an appropriate range ofto analysing a single series or multiple series, methods and don’t get side tracked into weight-whether it can run in automatic mode, the ing subsidiary dimensions over much. Thesophistication of the forecasting methods in- vendors (or consultants) will usually be willingcluded, its user interface, its simplicity, its link to develop appropriate report writers or datato external data bases and finally, price. For base features but without the core methods, theythose few systems I know well I felt the scoring have little of value to offer.was reliable.

Robert FildesThe report is designed to help select anThe Management School,appropriate forecasting system. Its layout is very

Lancaster University,helpful in this in that the summary will lead theLancaster LA1 4YX,user to a short list of packages (chapter 2). This

UKcan be further refined by reference to theindividual descriptions of each package (chapter

PII : S0169-2070( 00 )00091-13). Chapter 4 provides the detail sometimesneeded to determine the final short list. If theexpenditure is small, it would be sufficient to

Methodology and Tacit Knowledge: Two Ex-act as a buyer’s guide. The questionnaire isperiments in Econometrics, Jan R. Magnus andsurprisingly detailed on most aspects of soft-Mary S. Morgan (John Wiley, New York, 1999).ware design, so for example, particular attentionISBN: 0471982970, pp. 426, £55.is given to the estimation algorithms used. I was

particularly pleased to see that extensive cover-‘A master mystic was expected to burst forth from the skies,age was given to error measures from different

flashing for a short while like a meteor, then vanishing again.forecasting routines. (I note a key question Any records he left behind would be much read and admired

Book reviews 131

for a time, and remain not without influence on a small and leave food consumption unchanged (homogen-devoted community. But in the long run, all it had to hold on eity postulate)? Third, what methodologicalto was the original inspiration; everyone took from it what

reasons are responsible for getting results differ-suited him, and finally the community split up into factionsover the interpretation of his words.’ ent from Tobin’s results? Fourth, are the results

robust to other U.S. budget surveys? Fifth, whatEugen Herrigel, The Method of Zen

is the income elasticity using Dutch data?(1974, page 28).Finally, what is the expected food consumption

Differences in views among economists can for the Dutch economy through 2000?be resolved, in principle, through empirical To address these questions Magnus and Mor-tests. But how can one resolve differences in gan assemble various data sets: Tobin’s originalempirical tests when, literally, the same num- data set (1941 budget survey and time seriesbers are used to answer a given question? Are over 1913–41), three budget surveys for thedifferences in test results due to differences in United States (1950, 1960–61, and 1972–73)statistical methods or are they due to differences and relevant time series over 1912–1989,in implementation of a given method? This budget surveys for the Netherlands (1965, 1980,wonderful book will not give you unambiguous and 1988) and relevant time series over 1948–answers to these questions but it will unambigu- 1988. Thirty-seven research groups accepted theously question your answers. invitation but only eight of them completed the

The first two-thirds of the book is an experi- tasks. Reporting requirements conformed toment in finding out whether recent research in those of the Journal of Applied Econometricseconometrics contradicts the results of Tobin’s and included a log book recording the evolution1950 paper on the estimation of the income of the project. To evaluate the contributions,elasticity for U.S. food consumption. To this Magnus and Morgan rely on eight assessorsend, Magnus and Morgan extended an open who have made important contributions to bothinvitation to researchers to apply their methods economic theory and econometrics (A. Barten,and then compared their results. The answers, J.S. Cramer, A. Kapteyn, M. McAleer, H.one might suspect, differ and to isolate the Pesaran, P. Schmidt, K. Wallis, and M. Walkins).sources of those differences, the remaining third The assessors acted as referees for the Journalof the book is an experiment in finding out the of Applied Econometrics, which published therole of ‘tacit knowledge’: can an applied econo- papers of the first experiment.metrician offer persuasive research without any Chapter 2 is a reprint of Tobin’s paper.advice on how to implement a given method? Greatly simplified, Tobin used a cross section of

Chapter 1 outlines the details of the first family food expenditures in 1941 to estimate theexperiment. The material covers the questions income elasticity for food consumption in theexamined by the researchers, the data used as United States; his estimate is 0.56 (equation 8,input, the character of the communication page 29). Tobin then postulates an aggregateamong researchers and between the latter and demand function for food depending on incomethe experimenters (Magnus and Morgan), the and prices and estimates the associated pricereporting requirements, and the evaluation of elasticity subject to the constraint that the macroresults. Magnus and Morgan pose six questions. and micro income elasticities be the same.First, what is the income elasticity of U.S. food Based on U.S. macro data from 1913 to 1941,consumption using Tobin’s data? Second, will Tobin gets an own-price elasticity of foodidentical percent increases in prices and income consumption of 2 0.5 (equation 25, page 43)

132 Book reviews

and cannot reject the homogeneity postulate. nine-fold gap between the smallest and largestReading Tobin’s paper generates many ques- forecast.

tions: What if aggregation invalidates using a Given that these different answers rely on themicro-based estimate in a macro-based demand same data, Magnus and Morgan implement anfunction? What about the time-series properties experiment to find out if ‘tacit knowledge’of the data? What about heterogeneity among accounts for the dispersion of results. Theyfamilies, about censored data, about cross-price define (page 311) tacit knowledge as thatelasticities, about this and that? These are the ‘knowledge that fills the gap between methodo-questions that chapters 3–10 address. These logical treatises and successful applications.’chapters emphasize different aspects of the Chapter 15 details the design of this experiment:craft: time-series properties, heterogeneity and assigning to a master-level student the task ofsampling biases, model selection by statis- estimating the income elasticity of U.S. foodtically-based information measures, theoretical consumption with the methodologies of Leamer,considerations including choice of model and Hendry, and Sims. The results (chapter 16) arethe aggregation requirements to apply a micro then evaluated by Leamer, Hendry, and Paganrelation to macro data. Each chapter includes an (chapters 17–19) to see if the student masteredabbreviated log book, comments by three asses- the material as the masters intended. Chapter 20sors, and a reply from the chapter’s author(s). reports the interpretation of Magnus and Mor-

Chapter 11 offers Tobin’s own retrospective gan: From these three methodologies, only thaton his paper and on the idea of seeing whether of Leamer was mastered in the eyes of Leamer.its answers changed. Chapter 12 contains the The implementation of the other two methodsassessors’ views on the experimental design as fell short of what their masters intended. Thissuch and on the meaning of the results for gap between the ideal and the real could bemeasuring the progress of econometrics. Chap- interpreted as underscoring the importance ofter 13 compares the results across studies using tacit knowledge. That is, sole reliance on publi-well-designed tables and graphs. The resulting cations, with no advice on how to implement ainformation allows the reader to form an in- method, is not conducive to producing persua-dependent view of the differences across results sive research using that method. Nevertheless,as well as on how much each study accom- this interpretation is based on a sample of one.plished; no group addressed all six questions Finally, chapters 21–22 contain a completeposed by Magnus and Morgan. description of the data (variable names, defini-

Chapter 14 has the lessons that Magnus and tions, units, sources, and so on) and include theMorgan draw from this experiment. First, new web address for data retrieval, an offering thatresearch has not changed Tobin’s estimate of strengthens the already high quality of the book.the income elasticity for U.S. food consumption. Measuring the contribution of 50 years ofSecond, new research does not settle unambigu- econometric research involves unavoidable sub-ously whether the homogeneity postulate holds jective elements in the choice of both thefor U.S. food consumption. Third, new research reference point and the measuring rod. How-does not settle unambiguously whether the ever, readers would have benefited from know-income elasticity for U.S. food consumption ing whether the selection of Tobin’s paper wasvaries over time. Fourth, new research methods based on the number of citations or on thediffer greatly in their extrapolations of Dutch availability of data, what papers were rejectedfood consumption for the year 2000: there is a and why. What is at stake is not the quality of

Book reviews 133

Tobin’s paper but rather the robustness of the Forecasting Non-stationary Economic Timebook’s conclusions: would we reach the same Series, Michael P. Clements, and David F.conclusions if another paper, such as one of Hendry, The MIT Press, Cambridge, Massachu-those available in Hooper and Nerlove (1970), setts, 1999, ISBN 0-262-03272-4, US $35had been the reference point? As for the (Hardback)measuring rod, how can one measure the ad-vance of the field using papers from authors This is the second volume of two books onwho have developed the very field that is being macroeconomic forecasting. Vol. I was reviewedevaluated? in no. 3 (16) of IJF by myself and in JASA

If by progress we mean the degree to which (95), pp. 687–8, by Norman A. Swanson. Asquestions are settled, then the book offers compared to the title of Vol. I, the authors havecompelling evidence on how little econometrics added only ‘‘Non-stationary’’, but this generali-has progressed in the last 50 years. Indeed, that sation is more far-reaching than might be ex-the latest research cannot settle a central tenet of pected, because it also comprises deterministiceconomics, the homogeneity postulate, speaks breaks in time series.for itself. But thanks to Magnus and Morgan’s In Vol. II, the authors continue their inves-wonderful book, we know now how much work tigation into why model forecasts so often fail.is ahead of us. Few topics could be more essential to the

readers of this journal. The style of presentation1Jaime Marquez is again both elegant and efficient. The authors

Board of Governors convince the readers by first presenting theFederal Reserve System analytical results, then they use Monte CarloWashington, DC 20551 simulations to illustrate and finally they present

USA empirical examples where the analytical conclu-sions are shown to work.

The first two chapters are a repetition of themain results of Vol. I. This makes it possible to

References start the reading from Vol. II. The rest of thebook is about deterministic shifts in economic

Hooper, J. & Nerlove, M. (eds.) (1970). Selected Readingstime series: How do they affect forecasts, whatin Econometrics from Econometrica, MIT Press, Cam-happens when they interact with shortcomingsbridgeof the model and how to neutralise their effects

PII : S0169-2070( 00 )00071-6 using differencing and/or intercept correction(IC) or through modelling?

We learn some important lessons. A modelmay be correctly specified, but (after a shift) itmay still generate poor forecasts. Conversely, anon-causal model may forecast well. This result

1Federal Reserve Board. I am grateful to comments from suggests caution when judging forecast compe-Mary Morgan. The views in this paper are solely the titions. It also says that we may need one typeresponsibility of the author and should not be interpreted

of model for forecasting and another for policyas reflecting the views of the Board of Governors of theanalysis.Federal Reserve System or of any other person associated

Another fact that comes through very clearlywith the Federal Reserve System.