History of the Science of Modeling an Elephant
Post on 14-Oct-2014
DESCRIPTIONThis is a compilation of the amusing but enlightening scientific history on the question of how many parameters does it take to fit (or model) and elephant. We start with Freeman Dyson attributing the famous quote (regarding fitting an elephant) to John von Neumann by way of Enrico Fermi. There is then a vague attribution of this quote to several famous physicists in Brown and Sethna (2003) in Physical Review E. I then added Wei (1975) as the first somewhat serious approach to modeling an elephant. This is discussed in Burnham and Anderson (2002). Mayer et al. (2010) finally showed in their paper that indeed an elephant can be modeled with 4 parameters and with a 5th parameter its trunk can be made to wiggle. This paper was mentioned in Nature Physics in 2010, which also has a superior color presentation of Mayer et al.'s elephantine graph. Lastly, there was an amusing fictional intellectual history of modeling an elephant given by J. Dobaczewski in a presentation given at an actual scientific conference on nuclear structure physics in Finland.
essay turning points
A meeting with Enrico FermiHow one intuitive physicist rescued a team from fruitless research.Freeman Dyson
ne of the big turning points in my life was a meeting with Enrico Fermi in the spring of 1953. In a few minutes, Fermi politely but ruthlessly demolished a programme of research that my students and I had been pursuing for several years. He probably saved us from several more years of fruitless wandering along a road that was leading nowhere. I am eternally grateful to him for destroying our illusions and telling us the bitter truth. Fermi was one of the great physicists of our time, outstanding both as a theorist and as an experimenter. He led the team that built the first nuclear reactor in Chicago in 1942. By 1953 he was head of the team that built the Chicago cyclotron, and was using it to explore the strong forces that hold nuclei together. He made the first accurate measurements of the scattering of mesons by protons, an experiment that gave the most direct evidence then available of the nature of the strong forces . At that time I was a young professor of theoretical physics at Cornell University, responsible for directing the Crossed paths: A discussion with Enrico research of a small army of graduate Fermi (above) made Freeman Dyson students and postdocs. I had put them (right) change his career direction. to work calculating mesonproton scattering, so that their theoretical calculations a package of our theoretical could be compared with Fermis measure- graphs to show to Fermi. ments. In 1948 and 1949 we had made When I arrived in Fermis similar calculations of atomic processes,using office, I handed the graphs to the theory of quantum electrodynamics,and Fermi, but he hardly glanced found spectacular agreement between experi- at them. He invited me to sit ment and theory.Quantum electrodynamics down, and asked me in a is the theory of electrons and photons friendly way about the health interacting through electromagnetic forces. of my wife and our newBecause the electromagnetic forces are weak, born baby son, now fifty we could calculate the atomic processes years old. Then he delivered precisely. By 1951, we had triumphantly his verdict in a quiet, even voice. There are finished the atomic calculations and were two ways of doing calculations in theoretical looking for fresh fields to conquer. We physics, he said. One way, and this is the decided to use the same techniques of calcu- way I prefer, is to have a clear physical picture lation to explore the strong nuclear forces. of the process that you are calculating. The We began by calculating mesonproton other way is to have a precise and selfscattering, using a theory of the strong forces consistent mathematical formalism. You known as pseudoscalar meson theory. By the have neither. I was slightly stunned, but spring of 1953, after heroic efforts, we had ventured to ask him why he did not consider plotted theoretical graphs of mesonproton the pseudoscalar meson theory to be a selfscattering.We joyfully observed that our consistent mathematical formalism. He calculated numbers agreed pretty well with replied, Quantum electrodynamics is a Fermis measured numbers. So I made good theory because the forces are weak, an appointment to meet with Fermi and and when the formalism is ambiguous we show him our results. Proudly, I rode the have a clear physical picture to guide us.With Greyhound bus from Ithaca to Chicago with the pseudoscalar meson theory there is no2004 Nature Publishing Group
physical picture, and the forces are so strong that nothing converges. To reach your calculated results, you had to introduce arbitrary cut-off procedures that are not based either on solid physics or on solid mathematics. In desperation I asked Fermi whether he was not impressed by the agreement between our calculated numbers and his measured numbers. He replied, How many arbitrary parameters did you use for your calculations? I thought for a moment about our cut-off procedures and said, Four. He said, I remember my friend Johnny von Neumann used to say, with four parameters I can fit an elephant, and with five I can make him wiggle his trunk. With that, the conversation was over. I thanked Fermi for his time and trouble,and sadly took the next bus back to Ithaca to tell the bad news to the students.Because it was important for the students to have their names on a published paper, we did not abandon our calculations immediately. We finished them and wrote a long paper that was duly published in the Physical Review with all our names on it. Then we dispersed to find other lines of work. I escaped to Berkeley, California, to start a new career in condensed-matter physics. Looking back after fifty years, we can clearly see that Fermi was right. The crucial discovery that made sense of the strong forces was the quark. Mesons and protons are little bags of quarks. Before Murray GellMann discovered quarks, no theory of the strong forces could possibly have been adequate. Fermi knew nothing about quarks, and died before they were discovered. But somehow he knew that something essential was missing in the meson theories of the 1950s. His physical intuition told him that the pseudoscalar meson theory could not be right. And so it was Fermis intuition, and not any discrepancy between theory and experiment, that saved me and my students from getting stuck in a blind alley. Freeman Dyson is at the Institute for Advanced Study, Einstein Drive, Princeton, New Jersey 08540, USA.297
NATURE | VOL 427 | 22 JANUARY 2004 | www.nature.com/nature
HEKA DAVIS PHOTO/AIP EMILIO SEGR VISUAL ARCHIVES
UNIV. CHICAGO/AIP EMILIO SEGR VISUAL ARCHIVES
There is a famous aphorism in physics: Give me four parameters and I can fit an elephant. Give me five and I can wag its tail .  We have tried to find the appropriate attribution for this quote but have been unsuccessful. Variants of the statement (differing in the number of parameters) have been attributed to C.F. Gauss, Niels Bohr, Lord Kelvin, Enrico Fermi, and Richard Feynman. Excerpted from: Kevin S. Brown and James P. Sethna. 2003. Statistical mechanical approaches to models with many poorly known parameters. Physical Review E 68: 021904.
Excerpted from: Kenneth P. Burnham and David R. Anderson. 2002. Model Selection and Multimodel Inference: A Practical Information-theoretic Approach. 2nd ed. New York: Springer.1.4 Inference and the Principle of Parsimony 29
ously. The advent of Markov chain Monte Carlo methods (Gilks et al. 1996, Gamerman 1997) may soon give rise to a general but practical framework for spatiotemporal modeling; model selection will be an important component of such a framework. A step towards this general framework was made by Buckland and Elston (1993), who modeled changes in the spatial distribution of wildlife. There are many other examples where modeling of data plays a fundamental role in the biological sciences. Henceforth, we will exclude only modeling that cannot be put into a likelihood or quasi-likelihood (Wedderburn 1974) framework and models that do not explicitly relate to empirical data. All least squares formulations are merely special cases that have an equivalent likelihood formulation in usual practice. There are general information-theoretic approaches for models well outside the likelihood framework (Qin and Lawless 1994, Ishiguo et al. 1997, Hurvich and Simonoff 1998, and Pan 2001a and b). There are now model selection methods for nonparametric regression, splines, kernel methods, martingales, and generalized estimation equations. Thus, methods exist for nearly all classes of models we might expect to see in the theoretical or applied biological sciences.
Inference and the Principle of ParsimonyAvoid Overtting to Achieve a Good Model Fit
Consider two analysts studying a small set of biological data using a multiple linear regression model. The rst exclaims that a particular model provides an excellent t to the data. The second notices that 22 parameters were used in the regression and states, Yes, but you have used enough parameters to t an elephant! This seeming conict between increasing model t and increasing numbers of parameters to be estimated from the data led Wel (1975) to answer * the question, How many parameters does it take to t an elephant? Wel nds that about 30 parameters would do reasonably well (Figure 1.2); of course, had he t 36 parameters to his data, he could have achieved a perfect t. Wels nding is both insightful and humorous, but it deserves further interpretation for our purposes here. His standard is itself only a crude drawingit even lacks ears, a prominent elephantine feature; hardly truth. A better target would have been a large, digitized, high-resolution photograph; however, this, too, would have been only a model (and not truth). Perhaps a real elephant should have been used as truth, but this begs the question, Which elephant should we use? This simple example will encourage thinking about full reality, true models, and approximating models and motivate the principle of parsimony in the following section. William of Occam suggested in the fourteenth century that one shave away all that is unnecessarya dictum often referred to as Occams razor. Occams razor has had a long history
*Wei (1975); see previous.
FIGURE 1.2. How many parameters does does it take to t an elephant? was answered by Wel (1975). He started with an idealized drawing (A) dened by 36 points and used least squares Fourier sine series ts of the form x(t) 0 + i sin(it/36) and y(t) 0 + i sin(it/36) for i 1, . . . , N. He examined ts for K 5, 10, 20, and 30 (shown in BE) and stopped with the t of a 30 term model. He concluded that the 30-term model may not satisfy the third-grade art teacher, but would carry most chemical engineers into preliminary design.
in both science and technology, and it is embodied in the principle of parsimony. Albert Einstein is supposed to have said, Everything should be made as simple as possible, but no simpler. Success in the analysis of real data and the resulting inference often depends importantly on the choice of a best approximating model. Data analysis in the biological sciences should be based on a parsimonious model that provides an accurate approximation to the structural information in the data at hand; this should not be viewed as searching for the true model. Modeling and model selection are essentially concerned with the art of approximation (Akaike 1974).
Drawing an elephant with four complex parametersJurgen MayerMax Planck Instilflte of Molecular Cell Biology and Genetics, Pjotenilauerstr. 108, 01307 Dresden, Germany
Khaled KhairyEuropean Molecular Biology La.boratory, MeyerhofstrafJe. 1, 69117 Heidelberg, Gennany
Jonathon HowardMax Planck Institute of Molecular Cell Biology and Genetics, Pfotellhauerstr. 108, 01307 Dresden, Germany
(Received 20 August 2008; accepted 5 October 2009)
We define four complex numbers representing the parameters needed to specify an elephantine shape. The real and imaginary parts of these complex numbers are the coefficients of a Fourier coordinate expansion, a powerful tool for reducing the data required to define shapes. 2010American Association of PhYSics Teacllel'.r.
A turning point in Freeman Dyson's life occurred during a meeting in the Spring of 1953 when Enrico Fermi criticized the complexity of Dyson's model by quoting Johnny von Neumann:' ''With four parameters I can fit an elephant, and with five [ can make him wiggle his trunk." Since then it has become a well-known saying among physicists, but nobody has successfully implemented it. To parametrize an elephant, we note that its perimeter can be described as a set of points (x(t),y(t)), where t is a parameter that can be interpreted as the elapsed time while going along the path of the contour. If the speed is uniform, t becomes the arc length. We expand x and y separately2 as a
trace out elJiptical corrections analogous to Ptolemy's epicycles.' Visualization of the corresponding elJipses can be found at Ref. 6. We now use this tool to fit an elephant with four parameters. Wei' tried this task in 1975 using a least-squares Fourier sine series but required about 30 terms. By analyzing the picture in Fig. 1(a) and eliminating components with amplitudes less than 10% of the maximum amplitude, we obtained an approximate spectrum. The remaining amplitudes were
Fourier series x(t) = 2: (A% cos(kt) + B:l sin(kt)) ,k=O
yet) = 2: (A~ cos(kt) +B~ sin(kt)) ,k=O
100150 100 150
where Ai, 11k, Ak. and EX are the expansion coefficients. The lower indices k apply to the kth term in the expansion, and the upper indices denote the x or y expansion, respectively. Using this expansion of the x and y coordinates, We can analyze shapes by tracing the boundary and calculating the coefficients in the expansions (using standard methods from Fourier analysis). By truncating the expansion, the shape is smoothed. Truncation leads to a huge reduction in the information necessary to express a certain shape compared to a pixelated image, for example. Szekely et al.' used this approach to segment magnetic resonance imaging data. A similar approach was used to analyze the shapes of red blood cells: with a spherical harmonics expansion serving as a 3D generalization of the Fourier coordinate expansion. The coefficients represent the best fit to the given shape in the following sense. The k=O component corresponds to the center of mass of the perimeter. The k= I component Corresponds to the best fit ellipse. The higher order components648Am. I. Phys. 78 (6). Iune 20tO
120L. __ .. _.. _"'"_._ .... ______ .__ .... ,. __ .___ .......... __ ._.... _____ .1
Fig. 1. (a) Outline of an eJephnnt. (b) Three snapshots of the wiggling trunk.Q)
20 I0 American Association of Physics Teachers
Table I. The five complex parameters PI,'" ,Ps that encode the elephant including its wiggling trunk. ParameterPI::::50-30i Pl=18+8iP3=12-IOi
Real part81=50 ill=18
the special case of fitting an elephant, it is even possible to lower it to four complex parameters and therein implement a well-known saying.
ACKNOWLEDGMENTS Many thanks to Jean-Yvcs Tinevez and Marija Zanie, as well as the anonymous reviewers, for revising and improving this article.IF. Dyson, "A meeting with Enrico Fermi," Nature (London) 427(6972), 297 (2004). 2F. P. Kuhl and C. R. Giardina, "Elliptic Fourier features of a closed contour," Comput. Graph. Image Process. 18, 236-258 (1982). J G. Szekely, A. Kelemen, C. Brechblihler, and G. Gerig, "Segmentation of 2D and 3D objects from MRI volume data using constrained elastic deformations of flexible Fourier contour and surface models," Merl. Image Anal. 1(1), 19-34 (1996). 4K. Khairy and 1. Howard, "Spherical hlUTl1onics-based parametric deconvolution of 3D sutface images using bending energy minimization," Med. Image Anal. 12(2),217-227 (2008). sThe interactive Java applet written by Rosemary Kennett, (physics.syr.edu/courses/javaldemoslkennettJEpicyclelEplcycle.html). 6Interactive Java appJet of elliptic descriptors by F. Puente Le6n, (www.vms.ei.tum.de/Iehre/vrns/fourierf). ' J, Wei, "Least square fitting of an elellhant." CHEMTECH S. 128-129 (1975). 8See supplementary material at http://dx.doLorglJO.tl19/1.3254017 for the movie showing the wiggling trunk.
A)=-60 xcyc =Ycyc::::20
slightly modified to improve the aesthetics of the final image. By incorporating these coefficients into complex numbers, we have the equivalent of an elephant contour coded in a set of four complex parameters (see Fig. l(b)). Thc real part of the fifth parameter is the "wiggle parameter," which determines the x-value where the trunk is attached to the body (see the video in Ref. 8). Its imaginary part is used to make the shape more animal-like by fixing the coordinates for the elephant's eye. All the paranleters are specified in Table I. The resulting shape is schematic and cartoonlike but is still recognizable as an elephant. Although the use of the Fourier coordinate expansion is not new,2,3 our approach clearly demonstrates its usefulness in reducing the number of parameters needed to describe a two-dimensional contour. In
Am. 1. Phys . Vol. 78, No.6, June 2010
Notes and Discussions
research highlightsFitted at lastAm. J. Phys. 78, 648649 (2010)
In the spring of 1953, Freeman Dyson travelled to Chicago to meet Enrico Fermi. In his luggage, the young professor had the results of his groups long effort to calculate mesonproton scattering. But Fermi was not impressed, even if the agreement of Dysons calculations with experimental data obtained at the Chicago cyclotron was good. Fermi didnt trust the arbitrary parameters used in the calculations: I remember my friend Johnny von Neumann used to say, with four parameters I can fit an elephant, and with five I can make him wiggle his trunk. Dyson took Fermis point seriously, and changed his career direction. (I am eternally grateful to him, Dyson said half a century later, for destroying our illusions and telling us the bitter truth.) But still can the outline of an elephant be really defined with only four parameters? An early attempt at implementing that task with least-square fits failed. But Jrgen Mayer and colleagues have now succeeded, using appropriate truncated Fourier expansions. With four complex parameters, they describe a curve that can be interpreted as elephantine (pictured). And with a fifth parameter, they can control both the position of the trunk and the position of the eye.
polarization-entangled. The team had to take special care to ensure that which-path information wasnt available by another means measuring the energy of the photons, for example and that the cascade wasnt interrupted by any rogue third electron. The device has the advantage over the competing parametric downconversion approach in that the process is deterministic, potentially enabling the creation of an entangled pair at the press of a button. The work by Salter et al. is a beautiful combination of the two most important motivations for quantum-dot research: an electronic structure more complicated than natural atoms leads to a richer array of physics, in this case entanglement; and solid-state approaches promise compact devices that dont require a room full of expensive lasers.
reprinted with permission. 2010 AApt
which they can decay, they enhance the emission rate by increasing the density of these states. They have demonstrated this by depositing a layer of laser-dye atoms onto a forest of silver nanowires embedded in a porous alumina template a structure that has been predicted to behave as a so-called hyperbolic metamaterial with an anomalously high density of photonic states. The decay rate of the dye atoms increased by more than a factor of six. Such control is expected to be useful in the development of single-photon emitters.
Mon. Not. R. Astron. Soc. doi:10.1111/j.1365-2966.2010.16797.x (2010)
hyperbolic spontaneityOpt. Lett. 35, 18631865 (2010)
Spontaneity usually isnt easily controlled. One way to control an otherwise spontaneous process is through stimulation, as occurs in a nuclear chain reaction or a laser. Another is to change the environment in which the process occurs. This was one of the motivating concepts in the development of photonic crystals: if an excited fluorescent atom is placed in the centre of a material engineered to have no states for the atom to decay into, the atom should remain in its excited state indefinitely. The challenge has been to engineer a material with an effective and complete bandgap. Mikhail Noginov and colleagues have taken the opposite approach. Rather than reduce the rate at which fluorescent atoms emit by limiting the density of states into
At the centre of every galaxy is a supermassive black hole, surrounded by an accretion disk of gas and dust and possibly with jets streaming out of the plane of the disk. It had been thought that the faster the spin of the black hole, the more powerful the jets but then some fast-spinning supermassive black holes were found to have no jets at all. David Garofalo and colleagues think its not only about how fast the black hole spins, but in which direction. Black holes spinning in the same direction as their accretion disks prograde may indeed have no jets, whereas the strongest jets are more likely to arise from a galaxy that has a retrograde black hole, spinning in the opposite direction to its accretion disk. Garofalo et al. show that the key is the gap between the black hole and its disk, which can be large for retrograde black holes, but decreases as the black hole, accreting material from the disk over time, evolves towards increasing prograde spin. Jet production is most effective for a large gap, which is associated with strong black-hole-threading magnetic fields. Science 328, 12461248 (2010)
if a spin fluctuates in a forest
On-demand entanglementNature 465, 594597 (2010)
An entangled-photon-emitting diode has been created by Cameron Salter and colleagues. An applied bias injects two electrons and two holes into a single indium arsenide quantum dot. The electronhole pairs recombine in a cascade fashion that can follow one of two paths: the emission of a left-hand-polarized photon followed by a right-hand-polarized photon, or vice versa. As long as these two paths are indistinguishable, the two photons are478
A quantum spin liquid is a low-temperature state of an antiferromagnet in which quantum fluctuations prevent the spins ordering magnetically. The trouble is, how do you measure nothing? Several promising Heisenberg antiferromagnets show no magnetic order down to the lowest attainable temperatures but its not possible to reach absolute zero to be sure. Minoru Yamashita and co-workers have addressed this limitation by probing the lowenergy excitations of the organic insulator, EtMe3Sb[Pd(dmit)2]2, also known as dmit-131. Heat conduction is an excellent probe of low-lying excitations. Indeed, the extrapolated zero-temperature thermal conductivity of dmit-131 has a term that is linear in temperature, which is the signature of gapless excitations such as those found in a standard metal. Given the insulating nature of the material, it is surprising to find gapless excitations. Moreover, the mean free path exceeds 1,000 times the distance between the spins, consistent with ballistic propagation. This and other properties of dmit-131 suggest a quantum state that is distinct from the ordered states that we know.nature physics | VOL 6 | JULY 2010 | www.nature.com/naturephysics
2010 Macmillan Publishers Limited. All rights reserved
Arctic FIDIPRO-EFES Workshop Hotel Riekonlinna, Saariselk, Finland April 20-24, 2009 Future Prospects of Nuclear Structure Physics Workshop summary by J. Dobaczewski
Physics of elephantIn the second half of the twentieth century, physicists formulated the so-called elephant conjecture, also called Ises conjecture. It states that given enough parameters one can fit an elephant, and with some more one can make the elephant wag its tail. The true author of Ises conjecture is not known, however, the conjecture has been repeated many times at many conferences and finally got attributed to Dr. W. Ise of MIT, who was known for having formulated many interesting conjectures; so it seemed logical to attribute this one to him too. Initially, Ise did not object. Many were mystified by the conjecture, calling it first interesting, then challenging and fascinating, and at some point it has become one of the most important and burning questions in contemporary physics. The breakthrough came when a student of zoology S. Loppy, who was charged with transporting by plane an elephant to the Pittsburgh ZOO, forgot to properly lock the cargo door, and the elephant fell down from high altitude creating an impressive crater in the Mohave Desert. Feeling guilty, the student promptly traveled to the crash site and measured the diameter and depth of the crater to be 10.5 and 2.2 meters, respectively. His seminal paper on the First measurement of elephant by a high-energy collision was immediately accepted for publication in Physical Review Letters. Before the physics community had a chance to ponder on the significance of this measurement, Professor M. Ensa of Princeton, who met Loppy on his return flight to Pittsburgh, formulated the first model of the elephant, and managed to publish it back-to-back with Loppys article. The model described the elephant as a sphere of diameter of 7.2 meters and mass of 5 metric tons. These two parameters described Loppys data beautifully. A possibility of using similar experimental set-up to better establish properties of the elephant was immediately realized in several scientific centers. Important research programs were started, grants were awarded, and systematic observations of el-craters, as they were now called, were pouring into scientific journals. In experiments, the intensity of elephant beams was customarily measured in MA (mega amperes) with a proviso that the elephant before discharging was charged with one electrical shock of 110V during 1 second. Translation tables to units using European voltage of 220V were also published. A substantial grant awarded within the European FP137 program allowed for a construction of the multi-laser device (MLD) to observe whether the elephant is hitting the ground headfirst or tail-first. After a short, but heated discussion, about including the Physics of elephant in Physical Review E, which seemed like a very logical step, the APS opted for a creation of a new section F. One of the principal subsections in this journal was named Multifragmentation. With the accumulated large body of experimental observations, Ises conjecture could now be put to a stringent experimental test. The first model of the elephant, using very many parameters, was created within the research program sponsored by SONY, and described the surface of the elephant with a fantastic resolution of 72 dpi and later even 300 dpi. Millions and next billions of parameters were adjusted to the experimental results obtained for elcraters. Very advanced statistical methods to look at confidence levels were applied in collaboration with mathematicians, and finally it became clear that Ises conjecture has been proven false. The article on that issue entitled Ise proved wrong was published in the New
York Times. At this point Ise issued a statement saying that he, in fact, never formulated the conjecture. Since results of measurements of el-craters were made available on the Internet, many scientists and science aficionados attempted to confront the fascinating question: What the elephant really is? One of such persons was Dr. W.H. Ale who felt an intriguing affinity to the elephant and realized that Ensas model corresponded to the first term in a multipole series for elephants mass distribution. Very soon he was able to formulate an advanced model by using multipoles up to the order of 300 and next 500, which also contained millions of parameters. He invented his own judicious fitting procedure, adjusted elephants multipole moments to data and obtained good results. He was even able to understand the physical origin of the socalled elephant jets and proved that they must originate from the rear end of the elephant. Based on his analyses, Ale was able to put forward a daring hypothesis that four large heavy appendices protrude from one side of the elephant. In support of that, he produced fuzzy 3D computer-generated images of elephants mass distribution. Despite its success, Ales work has not gained much appreciation within the broad community, which seemed to be emotionally attached to Ises conjecture. His work was promptly qualified as just fitting many parameters, lacking physical understanding, or outright stupid. Ale tried to argue that the elephant is a fairly complex object and, therefore, precise description of high-resolution experiments must involve many parameters. To no avail; when the directors of experimental facilities announced: the elephant has been understood, the Nobel Committee honored the achievement and work of Ise, Loppy, and Ensa (Ise promptly issued a statement saying that he finally does recall having formulated his conjecture). Ale died in solitude after many years of untreated depression. However, his model is still widely used in the Ale Research Center, named after him, which now studies other large mammals. This paper has been presented in guise of a summary talk during the Arctic FIDIPRO-EFES Workshop held in Saariselk, Finland, on April 20-24, 2009, Future Prospects of Nuclear Structure Physics. The author gratefully acknowledges excellent working conditions at the Saariselk Ski Center, where most of the work has been done.