Pattern Recognition1966 IEEE WorkshopGeorge Nagy International Business Machines Corporation

Self-Organizing, Bionic, Heuristically Programmed, Reported development work on systems aimed atPattern Recognizing, Learning, Neuronal, Cybernetic, handprinted characters seemed to favor the on-lineGoal-Seeking, Problem-Solving, Microprogrammed, approach with immediate display of the interpretedMultiprogrammed, Multi-Input, Redundant, Adaptive, character, which allows the user to make on-the-spotSelf-Repairing, Self-Teaching, Time-Sharing, Self-Repro- corrections and to adjust his style to suit the recognitionducing, Cluster-Seeking, On-Line, Trainable, Stochastic, logic. The one off-line project described relies on theKilomegacycle, Optimal, Artificially Intelligent, Syn- context inherent in a programming language such asnoetic Computing Machines-these terms comprised one Fortran to keep the error rate within acceptable limits.speaker's list of key words necessary to describe the range Further contributions in character recognition con-of topics discussed at a recent "happening" (the chair- sisted of new algorithms designed to improve maximum-man's characterization) instigated by the Pattern Recogni- likelihood decisions based on features by taking intotion Subcommittee of the IEEE Computer Group. account the interfeature statistical dependences and theSome 52 pattern classifiers in search of recognition, Markovian properties of natural language.

divided evenly between private industry (26) and other Other applications-oriented presentations coveredcategories-universities (15), government agencies (5), holographic techniques for fingerprint recognition,and nonprofit laboratories (6)-attended sessions of the polynomial decision boundaries for electrocardiograms,Workshop on October 24-26, 1966, at the El Conquista- automated photometric blood cell analysis, adaptivedor and Dorado Hilton Hotels, Puerto Rico. The invita- networks for sonar phased antenna arrays and fortions had been mailed on the basis of recommendations aerial photoreconnaissance, a sequential decision modelby the members of the subcommittee as well as in re- for blackjack, graphic input for computers, the super-sponse to inquiries resulting from an announcement in position of flight paths on contour maps, and the analysisthe September 1966 issue of IEEE's Computer Group of three-dimensional projections. Among these, theNews. electrocardiogram analysis seems closest to practical

Fully a third of the formal presentations dealt with applicability. Several of the other projects, notably thesome form of character recognition, showing that this work on fingerprints, blood cells, sonar, and graphicendeavor still remains the most active single area in input, also make use of realistic data sets.pattern recognition. The characteristics of several com- The outline of a general-purpose pattern recognitionmercial print readers as well as specific applications, and manipulation system was also presented. Some ofsuch as the reading of social security forms, zip codes, the subroutines used for pattern segmentation and de-driver's license applications, editorial copy for type- scription are already operational.setting, and military allotment forms, were reviewed both Several of the theoretical papers described "unsuper-from the users' and the manufacturers' points of view. vised learning" schemes. A Fourier series expansion wasThe economic aspects of processing rejects and substitu- applied to the decomposition of multivariate normaltion errors were also discussed. distributions with a finite number of samples, and a

signal identification problem (additive noise, number ofGeorge Nagy is a rcsearch staff member, IBM Watson Re- messages unknown) was solved by means of a correlation

search Ccntcr, Yorktown Heights, N.Y. integral equation. Markovian statistics were drawn upon

92 IEEE spectrum FEBRUARY 1967

to investigate the convergence properties of an error- yields more insight than months of simulation. Nothingcorrecting training rule on a partially mislabeled sample is more fatuous than the current trend of correlatingset. everything with everything else, in the vain hope ofOther statistical contributions reported improved understanding obscure causal relationships.

estimation methods of a posteriori probabilities from Thesis: Computerized clustering techniques may proveadditional samples, and new bounds on the risk on the to be powerful aids in structuring the universe, andnearest neighbor decision rule with an infinite number of have already suggested improved classification schemessamples. to mathematical taxonomists in several branches of bi-The capabilities and limitations of two-layer threshold ology.

nets (simple perceptrons) were clarified in a paper linking Antithesis: The severest limitation encountered byrecognizable geometric properties with the maximum computer programs aspiring to intelligent activitynumber of connections to a unit of the first layer (the consists of the very restricted scope of knowledgeorder of the net). available to them. It is difficult to conceive of a clusteringAn interesting review paper described the difficulties algorithm, no matter how sophisticated, capable of

encountered in various approaches to "clustering," arriving at the fundamental distinction between verte-examined the relationships among the procedures in use, brates and invertebrates without the programmer'soffered standard distributions for the evaluation of having been aware of the importance of backbones as aclustering algorithms, and listed actual and potential feature for consideration at the outset.applications. The use of several varieties of two-dimen- Synthesis: Higher-order languages, leading to the de-sional projections of n-dimensional spaces of interest was scription, analysis, and synthesis of arbitrary patterns bysuggested as an aid in the study of multimodal distribu- computers, must continue to be developed. In the mean-tions. while, computers in pattern-recognition circles must earnOf interest to psychologists and engineers alike was a their rent by performing a variety of less glamorous tasks,

"grass fire" (self-propagating) global pictorial processing such as selecting "features" for character readers, verify-scheme designed to extract fairly abstract properties, ing assumptions and evaluating statistical parameters forincluding connectivity. Even longer-range strategy for specific data sets, working quickly through complicatedpattern recognition was represented by a report on studies logic trees and combinatorial calculations, facilitatingofhuman perceptual phenomena. man-machine interaction in engineering problems and

Philosophic advice from one quarter suggested that information retrieval, and testing hypotheses inefforts should be made to find "impotence conditions" perceptual psychology.in pattern recognition similar to those deduced in otherdisciplines. Context"Ask not what a pattern will do for you," urged an Thesis: The easy way to take advantage of context

advocate of the definition of suitable features through in natural languages is to use Markovian properties insynthesis rather than analysis. terms of bigram and trigram frequencies.

Since most of the papers presented are now awaiting Antithesis: Chomsky has shown that natural languagespublication, and since preprints of many are available are not Markov processes. Indeed, Markov processesfrom the authors, there will be no further attempt here to seldom occur in nature. Why not use syntactical rulessummarize each individual contribution. Rather, we combined with dictionary look-up techniques?shall now endeavor to review some of the issues that Thesis: Markov processes, in addition to providing angenerated discussion, argument, controversy, and excite- analyzable model to any required degree of approxima-ment. tion, do occur, as in thermal noise. A more complicatedThe matter of debate ranged from metaphysics model would not be tractable even with the use of high-

(pattern recognition as a problem in pattern recognition) speed computers.to practical detail (how to find the gap in C). The discus- Antithesis: The correction of errors in graphic inputsion here will follow a descending order of abstraction should occur at a level much higher than that of therather than a chronological sequence. symbols constituting the communication link between

man and machine. In graphic input as used for electronPattern recognition and the scientific method circuit design, for example, the machine should analyze

Thesis: Pattern recognition is the central problem of the system represented in terms of Maxwell's equationspsychology and philosophy. The general aim is to build rather than look for wiggly lines without connections toup a hierarchy of computer-recognizable patterns, both ends.somewhat parallel to human concepts, in terms of which Thesis: There are many levels of "well-formedness."any complexmay be analyzed. Adastraper aspera. Begin at the lowest usable level, that of the symbols,

Antithesis: Why use computers to perform "intel- instead of trying to teach the machine the complexitieslectual tasks?" At best the computer is a handy tool for of the real world.arithmetic, though even there its usefulness is often Antithesis: In many instances the lowest level of ab-overrated; a day or two with pencil and paper sometimes straction is not the most economical in terms of the

1966 IEEE Pattern Recognition Workshop 93

information to be transmitted to the computer. In "feature" for any quantity resulting from any dataSamuel's checker-playing program, for example, the processing between the original image and the finalmoves in the training games used to improve the pro- decision.gram's strategy are evaluated by the weighting algorithm The opinion was expressed that some attempt shouldof the program itself in order to detect keypunching be made to differentiate among various processes on theerrors. basis of the underlying mathematical formulation.

Synthesis: The introduction of context, at some level, The word "learning" appeared too pretentious for simpleis necessary in all but the simplest problems. The systems distance-minimizing algorithms; these, in turn, werepoint of view engendered by contextual considerations deemed a cut above "tracking" algorithms, operating onmay occasionally reveal that the task in hand is not, slowly varying systems, which avoided the problemafter all, best suited for pattern-recognition techniques. of local traps by remaining close to the real minimum at

all times.Parallel vs. serial processes

Thesis: The flexibility of stored-program digital com- Overviewputers more than offsets any advantages in speed that To a casual listener from another field of endeavor,may be obtained from special-purpose hardware. the claims of pattern recognition to the status of a

Antithesis: Analog methods, as in holography, can cohesive discipline, as substantiated by this Workshop,successfully attack correlation problems of a magnitude might have sounded a trifle exaggerated. Despite thenot even conceivable in terms of present-day general- definite community of interest among the participants,purpose equipment. evidenced by animated technical discussions in pool and

Thesis: In complicated problems the sample size is surf, at breakfast and over drinks, in rain forest andoften too small to allow training by example; in such casino, as well as in the meeting hall, it is clear thatcases, we must resort to training by description, which is heuristic methods have little to do with adaptive nets, oressentially a serial process requiring an adequate formal either of them with commercial print readers. To belanguage to convey the necessary heuristics. sure, psychologists willingly discuss the effects of retinal

Antithesis: In complicated problems the elements of stabilization with students of asymptotically efficientsolution are usually insufficiently well understood to estimators, and biologists interested in chromosomeallow the trainer to specify them explicitly to the ma- counts seek the advice of practitioners of holography,chine; thus, we must resort to adaptive nets and error- but this intercourse might seem to be more in the naturecorrecting algorithms, which are well suited to classifica- of a spirited conversation among intelligent people oftion tasks on poorly structured patterns. reasonably broad interests than an interchange of

Synthesis: The distinction between "serial" and "paral- technical information designed to advance specific re-lel," while superficially obvious, is difficult to define search goals.rigorously. Speaking loosely, it seems likely that systems This diversity of interests is particularly apparent todesigned for complicated pattern-recognition functions anyone attempting to organize a university course, gradu-in real-world environments will be parallel at the front ate or undergraduate, covering the general area of pat-end, and serial, or heuristic, at the back, with suitable tern recognition. A quick survey of existing coursesfeedback paths in between. Some biological and psycho- reveals little agreement as to the basic topics to belogical evidence appears to favor this view. included in such a course. Thus, perhaps fortunately, the

next generation of pattern recognizers is likely to be asFeatures and templates heterogenous a group as the present set.

Thesis: For multifont character recognition, whereseveral hundred distinct shapes are involved, templates Future concourseare impractical. From one point of view, the rather chaotic state of the

Antithesis: The extraction of features necessarily art only provides additional impetus for workshopsdegrades the information content of the images; hence, where the much heralded cross-fertilization can taketemplates are inherently superior. place. Then, as more and more recondite problems are

Thesis: Templates are too sensitive to noise when the tackled, truly interdisciplinary solutions will be developed,distinguishing feature between two classes is only a small with a fusion of the many interesting techniques nowfraction of the total area of the characters. being perfected in dark corners. This, at best, is the pious

Antithesis: There are more ways than one to skin a cat. hope; at worst, workers in different corners will discoverTo be successful, template matching must be preceded by that they are weaving the same web.adequate preprocessing techniques, including size nor- Several participants at the Workshop raised themalization, line thinning, random-dot elimination, and question as to what is proper material for presentationunskewing. at such a meeting. Should completed work be reviewed

even when already published elsewhere, with the objectDefinitions wanted of stimulating discussion? How numerous and how long

For performance specifications for commercial print should the formal presentations be and how much detailreaders, should agiven set of character shapes be referred on a given project is of interest to the whole group?to as a single "font" even when produced by different Would panel discussions on set topics be helpful? Doesprinting mechanisms? the overwhelming response to the call for ten-minuteShould the word "feature" be reserved for a distinct papers, with no advance abstracts required, indicate that

geometric subset of a pattern, or could it also be used to this formula will be as successful as in other disciplines?designate more complicated attributes such as connec- The IEEE Subcommittee on Pattern Recognitiontivity properties? Another suggested definition would use welcomes all suggestions.

94 1966 IEEE Pattern Recognition Workshop