Spline Smoothing and Nonparametric Regression. by Randall L. EubankReview by: Douglas NychkaJournal of the American Statistical Association, Vol. 84, No. 406 (Jun., 1989), pp. 623-624Published by: American Statistical AssociationStable URL: http://www.jstor.org/stable/2289968 .

Accessed: 16/06/2014 11:12

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

.

American Statistical Association is collaborating with JSTOR to digitize, preserve and extend access to Journalof the American Statistical Association.

http://www.jstor.org

This content downloaded from 195.78.108.37 on Mon, 16 Jun 2014 11:12:15 AMAll use subject to JSTOR Terms and Conditions

Book Reviews 623

ation and covariation. To overcome these problems, Yates proposes an integrated methodological approach to EFA wherein techniques for fit- ting the model, determining the number of factors, and rotating those factors are complementary to each other and focus on the central psy- chometric objective of identifying invariant and meaningful factors.

Yates's presentation of his perspective and methods is organized into 10 chapters. Chapter 1 provides a review of the algebraic and geometric representation of the common-factor model, with an emphasis on rela- tions of the model to multiple regression. The chapter is clear, but it is rather thin if viewed as an introduction to factor analysis, with limited coverage of basic concepts and principles.

Chapters 2-7 present the conceptual, mathematical, and geometric bases for the methodological cornerstone of Yates's approach to EFA- a technique for transforming factors so as to seek an invariant, simple- structure orientation. In Chapter 2, he discusses Thurstone's principle of simple structure and begins the development of a mathematical cri- terion for achieving optimal simple structure. He also begins to develop a theme that carries through subsequent chapters-that the popular con- ceptualization of simple structure and the rotational methods designed to achieve it have been misguided in that they focus on locating factors that represent clusters of variables, rather than on locating factors that define hyperplanes within which many variables lie. Throughout the rest of the book, Yates seeks to refocus simple-structure concepts and tech- niques on this objective, arguing that factors defined on the basis of coplanarity rather than colinearity of variables will be more invariant and meaningful. He does an admirable job of defining and defending this position in depth and employing it as the basis for building an in- tegrated methodology for EFA. He may, however, have overstated the degree of ignorance and neglect regarding this view of simple structure and the central objective of EFA and may not have given enough at- tention to other developments based on a similar general view (e.g., Cattell and Muerle 1960; Meredith 1977; Tucker 1970).

Chapter 2 culminates in the development of what Yates calls the "geomin" criterion for transformation, a criterion based on minimizing the sum of the geometric means of the squared loadings for all variables and incorporating a weighting scheme designed to provide for a tolerance of loadings that are sufficiently close to 0. Chapter 3 presents an illus- tration comparing geomin to direct quartimin that shows geomin to be more sensitive to the presence of factorially complex variables. In ad- dition, the notion of reducing the hyperplane width tolerance ("squeez- ing") during the iterative rotation process is introduced and found to be beneficial.

The central issue in Chapter 4 is the need to make assumptions about the nature of the factors to have a basis for obtaining an initial solution that can then be transformed to simple structure. Yates focuses on the assumption of a positive manifold-that is, that factors "may influence variables either independently or in conjunction, but do not tend to act at odds to one another" (p. 87). He relates this assumption to the work of Thurstone and the principle of simple structure, argues for its justi- fiability in many behavioral domains, and employs it as a basis for de- fining a starting configuration of test vectors. In particular, Yates suggests that the test-vector configuration should lie in a polyhedral cone ap- proximately centered on the major principal axis.

In Chapter 5, Yates modifies the geomin procedure so as to take into account the principles developed in Chapter 4. This includes incorpo- rating two types of weights into the mathematical criterion: (a) "distin- guishability" weights, designed to emphasize the role of all variables in the hyperplanes of the polyhedral cone, and (b) "complexity" weights, designed to emphasize variables that are nearly colinear with the major principal axis. These developments yield a general algebraic criterion that Yates calls "direct geoplane." In Chapter 6, Yates presents and refines this global strategy for transformation, and in Chapter 7 he pro- vides an illustration of direct geoplane using Thurstone's box problem. Chapter 7 also includes a presentation and illustration of the potential use of geoplane solutions to determine the number of major, invariant factors. This approach is based on the "extended vectors" method for inspecting the geometric configuration so as to distinguish between fac- tors lying at edges of the polyhedral cone versus those that merely rep- resent colinear clusters of variables.

In Chapter 8, Yates takes up the issue of fitting the common-factor model to data. He rejects purely statistical approaches because of their tendency to overfit the data and their misdirected focus on inferential issues. Instead he develops a weighted least squares technique wherein weights diminish the influence of superficial colinearities of highly similar variables. Chapter 9 provides a detailed illustration of this resistant fitting method along with geoplane rotation and the use of the extended vectors approach to help determine the number of factors.

In Chapter 10, Yates attempts to put his developments into historical

perspective. He reviews the controversy between Spearman and Thur- stone and laments the lack of a compromise, as well as the dominance of a line of methodology that grew out of a rather narrow and simplistic perspective on Thurstone's contributions. These points are emphasized via a presentation of a reanalysis of mental-testing data from Thurstone and Thurstone (1941) that shows a smaller number of invariant, mean- ingful factors than had been supported by the original investigators.

Yates closes with an epilogue in which he argues that EFA "has never been developed to anything approaching its full promise and potential" (p. 325) and that "only a radical reorientation of current perspectives will allow researchers to apply EFA in the manner envisioned by its originators" (p. 323).

I initially approached this book expecting an introductory treatment of EFA with some of Yates's specific views and techniques mixed in. The book is not like that at all. The presentation does not proceed through the conventional logical sequence corresponding to factor-anal- ysis steps of fitting the model, determining the number of factors, and transforming the solution. And there is very little discussion or presen- tation of conventional methods for doing these things. So the book is unlike any of the standard textbooks in this area (e.g., Gorsuch 1983; Harman 1976; Mulaik 1972), but I do think that Yates has been fairly successful at achieving the objective of presenting an alternative per- spective and methodology to users of popular factor-analysis techniques. It is important that readers keep in mind, though, that the central themes of his perspective are not really new (as he readily recognizes). He does, however, present a useful synthesis of these themes into a unified and coherent perspective on EFA and some methods that are technically sound and consistent with this perspective. Statisticians may not like his perspective much because of the priority given to psychometric issues over statistical ones, but the perspective is well presented, and it merits some attention.

ROBERT C. MACCALLUM Ohio State University

REFERENCES Cattell, R. B., and Muerle, J. L. (1960), "The 'Maxplane' Program for Factor

Rotation to Oblique Simple Structure," Educational and Psychological Mea- surement, 20, 269-290.

Gorsuch, R. L. (1983), Factor Analysis, Hillsdale, NJ: Lawrence Erlbaum. Harman, H. H. (1976), Modern Factor Analysis, Chicago: University of Chicago

Press. Meredith, W. (1977), "On Weighted Procrustes and Hyperplane Fitting in Factor

Analysis," Psychometrika, 42, 491-522. Mulaik, S. A. (1972), The Foundations of Factor Analysis, New York: McGraw-

Hill. Thurstone, L. L., and Thurstone, T. (1941), Factorial Studies of Intelligence (Psy-

chometric Monograph 2), Chicago: University of Chicago Press. Tucker, L. (1970), "Fitting of Factor Analytic Hyperplanes by a Personal Proba-

bility Function," technical report, University of Illinois, Dept. of Psychology.

Spline Smoothing and Nonparametric Regression. Randall L. Eubank. New York: Marcel Dekker, 1988. xvii + 438 pp. $89.75 (U.S.A.); $107.50 (elsewhere).

A regression analysis usually starts with specifying a parametric model that explains how the observed responses are related to the independent variables. In many situations, there is no scientific theory to suggest a functional form, and the model is often either specified from a casual graphical examination of the data or chosen for its convenience. The goal of nonparametric regression is to eliminate the subjective choice of a parametric model in cases in which information is not available. For example, given the additive model Yk = Pu(tk) + ek (1 c k ' n), where tk is an independent variable and ek is a random error, one would like to estimate the function y without making specific assumptions about its shape.

Most of the properties and descriptions of nonparametric methods for estimating functions are scattered throughout the statistical literature and often concentrate on the theoretical properties of these estimators. Eu- bank has succeeded in organizing this diverse literature into a book that could serve as an excellent textbook for a graduate statistics course on nonparametric regression or as an introduction for someone wanting to learn about this area. The book has seven chapters that group naturally into four major parts: Chapters 1-3 contain introductory material that draws some connections between parametric regression and regression on functions making up an orthogonal series (such as orthogonal poly- nomials); Chapter 4 presents kernel estimators; Chapters 5 and 6 cover

This content downloaded from 195.78.108.37 on Mon, 16 Jun 2014 11:12:15 AMAll use subject to JSTOR Terms and Conditions

624 Joumal of the American Statistical Association, June 1989

the theory and computation of smoothing spline estimates; and the last chapter presents least squares splines (regression on piecewise polyno- mials) and some other nonparametric methods.

This book reads easily, and Eubank has kept the mathematics to the level of advanced calculus and matrix theory. Although he includes some asymptotic theory, he balances these theoretical results with several good applications that illustrate the methods. The bibliography is extensive and the references helpful for directing interested readers to other sources. In addition, the problems at the end of each chapter often work out interesting details omitted from the text. One clear strength is the treatment of smoothing splines, a field that has long needed a unifying exposition that synthesized the numerous research articles. Besides con- sidering several extensions of the simple additive model, the volume also reviews recent work on confidence intervals and regression diagnostics for splines. This material is exciting, because it adds to a nonparametric estimate the same features of inference and model checking that are associated with ordinary parametric regression.

The main disadvantage of this volume is that besides being a textbook it also serves as a review of the most current literature. The result is many short sections that give a tantalizing introduction to a topic but then refer readers to another publication for details. I also have some minor criticisms about the content. Given the thorough treatment of kernel and spline methods it seems redundant to devote nearly a whole chapter to least squares splines. A better use of this space would be to give more details on nonparametric estimates in the context of gener- alized linear models and on robust procedures. In particular, the use of an iteratively reweighted smoothing spline algorithm to compute these estimates should be covered. From a theoretical point of view, the ex- amples of asymptotic results rely too heavily on Fourier analysis. Ex- panding a spline estimate as a trigonometric series is only useful when the true function, u, is periodic and the data are observed at equally spaced points. Because this book emphasizes cross-validation for se- lecting the smoothing parameter (or bandwidth), the author should have included some of the theory for the cross-validation function. For ex- ample, it is straightforward to analyze the limit of the expected value of the cross-validation function as n -x c. Deriving some of this function's properties may make cross-validation seem less mysterious.

At this point some readers might be thinking, "Well, this sounds like a good book, but I still do not know what a spline is." Here is one short description. (See also Silverman 1985.) A spline is a flexible strip of metal that was used by draftsmen (before the advent of computer-aided design systems) to connect a series of points with a smooth curve. The mathematical formulation of a spline attempts to capture some of the features of this physical device. In the simplest case, a spline estimate of ,u will minimize the sum of squared residuals subject to the constraint that the estimate can not be too rough or "wiggly." This constraint can be thought of as limiting the amount of work one can do in bending the metal strip to fit the data. If this strip has some flexibility it will adjust for points that deviate moderately from a smooth function but can- not accommodate very sharp bends to fit extreme points. Mathemati- cally, for some M < X a spline estimate is the function that minimizes Ek =I l Yk - h (tk)]2 subject to f h"(t)2 dt c M. Here the integrated, squared second derivative is a measure of the overall roughness (or the total bending energy stored in the metal strip). Because there are many dif- ferent observational models and ways of quantifying the roughness of a function, there are also many different splines. Perhaps the best known is an estimate that takes the form of a piecewise cubic polynomial, where the join points of the polynomial segments are at the observation points, tk. This form, however, does not hold for all splines. The real common denominator is that all of these estimators are solutions to constrained minimization problems. Surprisingly, these curve estimates can often be computed as efficiently as those based on local averages.

One advantage of spline estimates is that it is easy to formulate esti- mates for more complicated situations than just a simple additive model. Although kernel estimates of a regression function are easy to define, it is difficult to extend these estimates to situations in which locally weighted averages are no longer appropriate. Take for example the semiparametric model Yk = Zkfl + u(tk) + ek, where Zk is an additional covariate that is linearly related to the dependent variable by the parameter ,. The spline estimate is simply the result of minimizing the residual sum of squares over both p and fl (with the roughness constraint on p). In contrast, it is not obvious how to estimate p and ,B using kernel methods. Spline estimates also have a Bayesian interpretation in terms of a pos- terior mean. This property has led to some approximate confidence intervals for the spline estimate based on the posterior variance.

Early research on nonparametric regression focused on asymptotic convergence rates for simple models. This book gives an overview of a

field that has matured into a practical set of methods for data analysis. In the future, nonparametric techniques will complement or even replace standard parametric regression in some applications, and I believe that this book will contribute to this broadening of statistics.

DOUGLAS NYCHKA North Carolina State University

REFERENCE

Silverman, B. W. (1985), "Some Aspects of the Spline Smoothing Approach to Non-parametric Regression Curve Fitting" (with discussion), Journal of the Royal Statistical Society, Ser. B, 47, 1-52.

Robust Statistics. Frank R. Hampel, Elvezio M. Ronchetti, Peter J. Rousseeuw, and Werner A. Stahel. New York: John Wiley, 1986. xxi + 502 pp. $39.95.

Along with Huber's (1981) book of the same title, Robust Statistics is required reading for anyone involved in robustness research or desiring to start a research program in robustness. Here "robust statistics is a body of knowledge, partly formalized into 'theories of robustness,' re- lating to deviations from idealized assumptions in statistics" (p. 6). Ham- pel, Ronchetti, Rousseeuw, and Stahel give us a comprehensive book, from the basic ideas of robust statistics to advanced research problems. By the end, you will have seen a Sears catalog of "influence functions," "change of variance functions," and "breakdown points."

The book begins and ends with relatively nontechnical essays on ro- bustness. These two chapters (1 and 8) are good reading for everyone. The biases of the authors are easy to spot and are enjoyable, if not always believable. Section 8.2, "Some Frequent Misunderstandings About Ro- bust Statistics," is particularly entertaining. Section 8.1 deals with one of my pet peeves-namely, that the most frequently violated, but most frequently ignored, assumption is that of independence.

In Chapters 2-7, the authors develop the mathematical formulation of robust statistics. Chapter 2 presents location-parameter estimation, which is a problem most statisticians associate with robust methods. Chapter 3 treats the corresponding single-parameter testing problem. Chapters 4 and 5 cover estimation for multiple parameters, including multivariate location parameters and covariance matrices. Chapters 6 and 7 develop robust estimation and testing methods for linear models.

As a nonrobust researcher, I found the middle chapters a bit rough. The notation is formidable, although probably crystal clear to the every- day robust researcher. There are many references to numbered equations many pages away with no hint in the text as to what is being referenced. The data sets are small and simply too "standard" to be compelling. I doubt that this will be the last time we see the "stackloss data" of Brownlee (1965). I think the promise of robust statistics is to offer meth- ods useful in an automated setting with large data sets. Eddy and Kadane (1982) presented such a setting and applied robust regression to estimate the cost of drilling for petroleum. There is simply not enough clear practical information in the book to fulfill the goal of the preface, "to get everyday's robust methods going" (p. xi).

KINLEY LARNTZ University of Minnesota

REFERENCES Brownlee, K. A. (1965), Statistical Theory and Methodology in Science and En-

gineering (2nd ed.), New York: John Wiley. Eddy, W. F., and Kadane, J. B. (1982), "The Cost of Drilling for Oil and Gas:

An Application of Constrained Robust Regression," Journal of the American Statistical Association, 77, 262-269.

Huber, P. J. (1981), Robust Statistics, New York: John Wiley.

Application of Structural Systems Reliability Theory. Palle Thoft-Christensen and Yoshisada Murotsu. Berlin: Springer- Verlag, 1986. viii + 343 pp. $58.

This work is a follow-up to Structural Reliability and Its Applications (Thoft-Christensen and Baker 1982). The first book dealt with funda- mental principles of structural-reliability theory used to evaluate indi- vidual components such as beams or columns under the influence of

This content downloaded from 195.78.108.37 on Mon, 16 Jun 2014 11:12:15 AMAll use subject to JSTOR Terms and Conditions