Math/ Compur. Modrlhng. Vol. I I, pp. 1 5, 1988 Printed in Great Britain

PLENARY LECTURES

0X9.5-71 77% $3.00 + 0.00 Pergamon Press plc

ARE ECONOMIC FORECASTS ANY GOOD?

Murray L. Weidenbaum and Stephen C. Vogt

Center for the Study of American Business, Washington University, Campus Box 1208, St. Louis, Missouri 63130 USA

Abstract. An important part of the work of economists is to provide decisionmakers with usable forecasts of future economic activity. Economists have come under repeated criticism for their lack of ability to do so. This paper uses Livingston Survey data on consumer prices and industrial production to test for predictive accuracy over four decades beginning in 1947. Additionally, the short-term track record of computer-based econometric forecasts is analyzed by using a consensus of such forecasts published by Blue Chip Economic Indicators. The results are mixed. Depending on the series used, economists' ability to predict real variables such as industrial oroduction and real GNP aooears to have improved. The data on inflation a;e far less supportive of'this conclusion. The highly volatile inflation rates experienced since the mid 1970s only partially explain the poor performanie of inflation forecasting.

Keywords. Forecasting, economics, Livingston Survey, Blue Chip Economic Indicators

INTRODUCTION

Before presenting our more serious analysis, we will begin by launching what is known in the military-industrial complex as a preemptive first strike. Our target is all those silly economist jokes that infest the universe of non-economists.

We intend to demonstrate at the outset that our collection of economist jokes is far superior to the personal inventories of non-economists. To begin, we readily acknowledge that economics is the only profession that has gone downhill from the beginning. After all, the very best economic forecast was made a long time ago. As many of us recall from our Biblical studies, Joseph predicted a 14-year business cycle -- seven good years followed by seven lean years. And he hit it on the head.

We must be candid. With all the advances in economic analysis, mathematical modeling, and computer capability, Joseph's forecasting record stands unequaled. That is why our favorite wisecrack is that, if all economists in the world were laid end to end, it would be a good thing.

But in good Biblical terms, our task is not to don sackcloth and ashes or to recite from the Book of Lamentations. Rather, our purpose is to explain why decisionmakers in both business and government continue to find economic forecasts so useful.

Our criterion of usefulness, fundamentally, is an economic approach. We cite the Inany millions of dollars that continue to be spent each year to perform or acquire forecasts of

the economies of the United States and of the other major industrialized nations. Surely, any activity that helps to foster full employment among economists deserves the authors' enthusiastic endorsement.

But, of course, every profession must have some outlet for fun and games. For economists, it has become forecasting the economic growth rate and the pace of inflation for the next quarter of the year. Because of the magnitude of resources devoted to the task -- and the widespread dissatisfaction with the results -- we feel compelled to present a warning about pinpoint accuracy. Some years ago, one of our neighbors in St. Louis remarked that the Mississippi River was exactly 1,000,008 years old. How could he be so sure? It turned out that just eight years previously he had heard the state geologist set 31d Muddy's aye at 1 lnillion years.

Candidly, we must acknowledge the ability of the economics profession to make quarterly forecasts of the national economy is on par uith our neighbor's capability to pinpoint the age of the Mississippi. Modern computers can generate numbers far more rapidly than our capacity to use then for making valid economic predictions. Time and time again in recent years, projections of economic growth and inflation for the next three months have been not only off in ldagnitude but even in terms of the direction of change. And these forecasts on occasion have been off the mark so substantially as to be worse than useless. Thus, these very short-term have been misleading to the users and destructive to the general reputation economists.

estimates

of

There is much that economists do know about how an economy operates. For one thing, there is nearly universal agreement in the economics profession on microeconomic matters that is not appreciated by the public. For example, clamp a ceiling on rents (albeit from the most altruistic motive) and you will produce a shortage of housing. Indeed as a general proposition, artifically depress the price of any good or service and you will soon find that more people will want to buy and consume the item than are willing to produce and sell it at that price.

That elementarv bit of economic understandinq. the basic law bf supply and demand, continue;. to have powerful applications in policy making. Those apblications'range from the.effects of price controls to the impacts of farm price supports to the ramifications of quotas and tariffs on imports.

accohnting for a great number of Imonetary, fiSCd1, industrial, international, and other influences.

At the macroeconomic level, the results of any one policy action -- such as cutting income tax rates or slowing down the growth of the money supply -- are harder to gauge. The basic reason is that so many other factors are at work at the same time. Unlike experimenters in the physical sciences, economists (and other social scientists) cannot hold everything else constant. There is no laboratory where economists can perform repeated experiments. Rather, they must rely on empirical data where there are no control variables. Forecasting the oerformance of a national econolnv reouires

5 percent or more in 1984. The expansion for the year reached more than 6 percent.

Even more recent experience was similar and also positive. Over the last two years, tmost forecasters have been predicting economic growth considerably below that of 1984's rate -- and we surely delivered as growth slowed to 2.7 percent in 1985 and 2.5 percent in 1986 (Economic Report Report of the President, 1987). Like neighbor in St. Louis, those who anticipate oinooint accuracv will be disappointed. But preiailing econoMic forecasts have been helpful in indicating the general direction and order of Imagnitude of economic changes for the year ahead.

A MORE FORMAL ANALYSIS

econometric theory,

4 cvnic once said that economic forecastinq is neither an art nor a science -- it is a haiard.

a; well'as advance; made in

In this spirit, let us examine more formally the track recbrd of economic forecasters and see if tnis criticism is valid. First, we will analyze the extent to which economic forecasting has

the practical application of forecasting

improved over time.

techniques.

The evidence, although mixed, seems to indicate that forecasting performance has improved only marginally. Next, we will discuss the absolute success (or failure) that economic projections have achieved in recent tilnes. The findings are d bit more positive on thdt score. Finally, let us discuss what the future holds for making economic forecasts -- the Drosoects presented by

Despite its complexity, economists continue to make progress on matters relating to economic forecasting. We actually do learn from experience, in an elementary but useful feedback effect. Empirical evidence influences the development of new theories that explain the underlying structure of the economy from which, in turn, we make our projections. Looking over the past few years, we contend that annual forecasts of key economic variables have, on balance, served a useful purpose for business and government decisionmakers.

Indeed, let us go from theory to practice and examine the recent historical record. In late 1982, for example, most professional fore- casters projected a substantial increase in the economy's growth rate -- a shift from a decline of 2 percent in 1982 to d positive expansion in the neighborhood of 3 percent in 1983. We need to keep in mind the annoying fact that successive revisions in the data complicate any comparison of forecasts and actuals. Historical economic statistics are revised so frequently that one wag lamented that: "Tne past is as uncertain as the future."

Nevertheless, the actual growth in 1383 of 3.7 percent was not fundamentally off tne mark. Thus, economic forecasters in 1982 were essentially correct in projecting that 1983 would be a period of reasonable growth compared to the recession in 1982. Those who relied on

Since such an enormous amount of resources has been devoted to strengthening forecasting and forecasting Imethods, the critic is entitled to ask what advances hdve been made over the years. Certainly, from a technological standpoint, we should expect a vast improvement. The advent of the "computer age" has created many opportunities for more sophisticated Imodels, more precise "number crunching," and faster and rnore frequent forecasting. We are trying to avoid relying on that hoary old rule for success -- or at least survival: forecast frequently.

Important reasons obtain for contending that accurate macroeconomic modeling has become Imore difficult. While advances in theory and technoloyy have occurred, they have been outpaced by the incredsed complexity of our economy, and especially its increasing openness in a truly dynamic world economy. Such factors as overall size, rapid technological change, deregulation, and the share of income dnd output that is determined by goverrmient and not lnarket forces also influence the structural parameters of our lmodels. Those forces make modeling and forecasting considerably more challenging. Which of these forces has dominated tnrouyh time is anyone‘s guess. But, surely econolaics and econometrics are waging a ferocious battle.

is to calculate the mean absolute errors of those forecasts with the actual outcomes.

A rough, but hopefully enlightening, approach to evaluating the ilnprovement of economic forecasts

that projection were not misled.

Likewise, the popular projection in the fall of 1983 was reasonably helpful. Forecasters gen-

Perhdps the longest running historical series of

erally anticipated a substantial acceleration forecasts for the American economy is Joseph

in the growth rate from 3.7 percent in 1983 to Livingston's survey of economists' predictions of consumer prices, which has been in existence

Proc. 6th Int. Cmf. on Mathematical Modelling 3

since 1946.

Furthermore, since it is a composite forecast, the Livingston survey forces a consensus among surveyed prognosticators and acts as a reasonable proxy for the overall quality of economic prognostications. While that survey is not composed entirely of the results of technical econometric models, the proportion represented by formal modeling techniques has increased over the years as such models have grown in number and popularity.

Have economists learned to forecast better? Results from the Livingston data suggest that we have -- but only slightly. Let us examine the results. Table 1 reports the mean absolute forecast errors for the six-month ahead forecasts during each of the four decades that the Livingston Survey has been in existence.

TABLE 1 Forecast Record on Inflation, 1947-1986 _______________________----____________________

Coeffi- Weighted Mean cient of Mean-Fore-

Period Inflation Error Variation cast Error ____________________~~~~~~~~~__________________ Dec. 1947- June 1957 2.3% 3.2% 1.7% 1.9

Dec. 1957- June 1967 1.9 1.0 0.5 2.1

Dec. 1967- June 1977 7.0 2.8 0.4 7.3

Dec. 1977- Dec. 1986 7.6 2.2 0.6 3.6 _______________________________________________

Source: Forecasts of inflation were compiled from the annual survey by Joseph Livingston reported in the Philadelphia Inquirer, 1947-1986.

As would be expected, the mean forecast error is significantly larger in the more recent time periods. The forecast error for the period 1977-1986 is 2.2 percentage points, compared with a mean error of 2.8 percentage points for the period 1967-1977 and 1.0 percentage points for 1957-1967. Generally higher rates and volatility of inflation over the two most recent time periods alone may explain the higher forecasting errors.

A more aoorooriate conoarison between these two periods ;; tb weight the forecast errors relative to the coefficient of variation of actual inflation rates. Comparing these weighted forecast errors shows a reduction of over one-half in the 1977-1987 period relative to 1967-1977. Furthermore, the'weighted medn forecast error for 1967-1977 (7.3) appears somewhat anomalous when compared with the weighted errors for all four decades. Not surprisingly, it was over the 1967-1977 time frame that criticism of economic forecasting accelerated.

Some evidence on the ability of forecasters to project real economic variables is also avail- able in the Livinoston data. Real GNP fore- casts were not requested by Livingston until 1971. However, a long enough time-frame is provided by the estimates of industrial prod- uction, which have been reported since 1951.

This series has been used to analyze economists' ability to forecast real economic-variables (Keen, 1987). Howard Keen, Jr. reported averaqe absolute forecast errors for the twelve and six month ahead forecasts for the four periods 1951-1959, 1960-1969, 1970-1979, and 1980-1985. His results are presented in Table 2. The average error for the first period was 4.3 percentage points. The errors fell to 2.8 and 2.0 percentage points in the second and third periods respectively, indicating an improvement in the ability to project industrial production. But, the trend toward improvement has come to an abrupt halt in the most recent period. The average absolute forecast error for the period 1980-1985 rose to 4.1 percentage points, doubling the average error in the previous decade.

TABLE 2 Forecast Record on Industrial Production. 1951-1985

Industrial i4ean cient of Mean-Fore- Period Production Error Variation cast Error ________________________--_-____________________ 1951-1959 2.5% 4.3% 2.6% 1.9

1960-1969 3.3 2.8 1.1 3.0

1970-1979 2.2 2.0 2.3 1.2

1980-1985 1.8 4.1 2.7 1.9 ________________________-------_________________ Source: Business Economics, January 1987,

PP. 38-39.

As in the case of inflation, between 1967 and 1977, a major reason for the increase in forecast errors is the increased volatility of the industrial production series in recent years. Again, if we weight the lnedn forecast error by the coefficient of variation of the actual series, the results change substantially. The weighted mean forecast error in the current period is the second lowest of the four periods, being bettered only by lower errors in the 1970-1979 period.

A generous interpretation of the Livingston data yields at best the conclusion that there has been a slight improvement in economists' ability to forecast inflation and industrial production. But, given the resources devoted to this task, progress has been exceedingly slow.

THE CURRENT FORECASTING ERA

Even though progress toward better forecasting has lagged, it is important to evaluate the degree to which current-day forecasts have been on track since such projections are used heavily by both business and government decisionmakers. A relevant question to ask is "Do these forecasts allow people to make better decisions, or would they have been better off using some ad hoc means of projection?"

To dnswer that ouestion. let us examine the composite forecdsts of real economic growth and inflation by the 50 leading professional economic forecasters that comprise the Blue Chip Economic Indicators. Over the past decade, the monthly Blue Chip has become the most widely used proxy for the prevailing economic forecast

of the private sector. For the last ten years, let us compare the Blue Chip data with an alternative naive trend forecast. The naive trend forecast simply assumes that the rate of change this year will be the same as last year's (Eggert, 1987).

The results of such an analysis are found in Tables 3 and 4. Those decisionmakers who relied on Blue Chip for a GNP growth forecast were not let down. Over the ten years for which the data are available, the average absolute forecast error for Blue Chip was 1.0 percent. For the naive forecast, the average error was substantially higher -- 2.6 percent. Furthermore, the Blue Chip forecasters were consistent. Only in 1978 did the consensus of economists forecast GNP growth more poorly than the naive trend approach.

TABLE 3 Blue Chip vs. Naive Trend Method: A Companson in torecasting Accuracy for Percentage Change in Real GNP

_______________________________________________

Forecast Blue Chip Naive Error Forecast Consensus Trend Blue Error

Year Forecast Forecast Actual Chip Trend _____________________-______________________-__

1977 4.9% 4.9% 4.7% +0.2 +0.2

1978 4.3 4.7 5.3 -1.0 -0.6

1979 2.7 5.3 2.5 +0.2 +2.8

1980 -0.2 2.5 -0.2 0.0 +2.7

1981 0.9 -0.2 1.9 -1.0 -2.1

1982 2.2 1.9 -2.5 +4.7 +4.4

1983 3.2 -2.5 3.6 -0.4 -6.1

1984 5.1 3.6 6.4 -1.3 -2.8

1985 3.5 6.4 2.7 +0.8 +3.7

1986 3.1 2.7 2.5 +0.6 +0.2 --

iqean Absolute Forecast Error 1.0 2.6 ____________________-__-___-______-____________

Source: Challenge, July-August 1987, p. 60.

However, predictions of the inflation rate were less successful for the consensus. The leading professional forecasters performed a bit Norse, on average, than the naive trend results. The mean absolute percent error for the Blue Chip projection was 1.2 percent over the past decade, compared to the 1.0 percent average error for the naive trend. Also, the consistency of Blue Chip that we found in the numbers on real growth is not apparent in the inflation figures. Only in three of the ten projections made over the lo-year period did the consensus forecast outperform the trend. On the positive side, economists were able to predict more accurately the reversal of inflation in 1981-82 (Eggert, 1987).

The data presented here do not represent re- sounding support for adhering uncritically to eCOnOmiC forecasts, particularly with regard to inflation projections. Consequently, it should not be surprising that government policy-

TABLE 4 Blue Chip vs. Naive Trend Method: A Comparison in Forecasting Accuracy for Percentage Change in GNP Pnce Deflator

________________________~~~~~~_~~~_~~__~~__~___ Forecast

Blue Chip Naive Error Forecast Consensus Trend Blue Error

Year Forecast Forecast Actual Chip Trend __________________________-__-____-____________

1977 5.6% 6.4% 6.7%

1978 6.0 6.7 7.3

1979 7.3 7.3 8.9

1980 8.5 8.9 9.0

1981 9.1 9.0 9.7

1982 7.8 9.7 6.4

1983 5.7 6.4 3.9

1984 5.0 3.9 3.8

1985 4.7 3.8 3.3

1986 3.8 3.3 2.7

Mean Absolute Forecast Error

-1.1 0.3

-1.3 -0.6

-1.6 -1.6

-0.5 -0.1

-0.6 -0.7

t1.4 t3.3

+1.8 +2.5

t1.2 to.1

t1.4 to.5

t1.1 to.6

1.2 1.0 _________________________-_____________________ Source: Challenge, July-August 1987, p. 61.

makers, business executives, and investors generally have learned to take such forecasts with a grain -- or more of salt.

PROSPECTS FOR ECONOMIC FORECASTING

As in other areas of human endeavor, hope springs eternal. Despite the rather uneven performance recorded in recent years, bright spots are appearing on the economic forecasting horizon. Since the mid-1970s, a great deal of fruitful debate on the issue of econometric modeling has occurred. In part the criticisms of traditional economic theory and its applications made by the students of the Rational Expectations school have served to focus attention on the shortcomings of more traditional models (Lucas, 1984). Idhile the practical alternatives offered by this new school of thought have by themselves not met with any particular success, the deoate on economic modeling has moved to a higher plane.

As a result of this debate, several avenues of research in econometric modeling are now evolving. Those researchers more sympathetic to reduced form equations have focused their energies on vector autoregressions and especially Bayesian vector autoregressions as a Imeans of lnaking both conditional and unconditional forecasts of economic variables (Todd, 1984 and Litterman, 1984). Furthermore, tnese techniques have enriched the debate over economic causality and, in turn, provided information for theorists and practioners alike.

Economists whose proclivities are toward larye scale structural models have become more conscious of public expectations when offering tneir projections of economic variables. In Particular, how to properly incorporate these

expectations into such models in a meaningful way is very much a subject of ongoing research. To some degree, bigger is better. These structural models are continually being expanded and becoming more detailed in an effort to provide greater realism in describing economic processes.

CONCLUSION

Theoretical and practical applications for improving economic forecasting continue to be discovered by researchers in the field of economic model building. A promising technical session of this conference is devoted to enhancing the dialogue on continuing research in economic forecasting.

While a great deal remains to be contributed to the task of improving economic projections, it is important to reflect on their usefulness. Despite the shortcomings that we have noted, economic forecasts play an important role in the real world.

Consider the history of business forecasting in the United States. In the late 196Os, when