ADVANCES IN NEURAL INFORMATION

PROCESSING SYSTEMS 10

Proceedings ofthe 1997 Conference

edited by

Michael I. Jordan, Michael J. Keams and Sara A. Solla

A Bradford Book The MIT Press

Cambridge, Massachusetts London, England

Contents

Preface xiii

NIPS Committees xv

Reviewers xvii

Part I Cognitive Science

Synchronized Auditory and Cognitive 40 Hz Attentional Streams, and the Impact ofRhythmic Expectation on Auditory Scene Analysis, Bill Baird 3

On Parallel versus Serial Processing: A Computational Study of Visual Search, Eyal Cohen and Eytan Ruppin 10

Task and Spatial Frequency Effects on Face Specialization, Matthew N. Dailey and Garrison W. Cottrell 17

Neural Basis of Object-Centered Representations, Sophie Deneve and Alexandre Pouget 24

A Neural Network Model of Naive Preference and Filial Imprinting in the

Domestic Chick, Lucy E. Hadden 31

Adaptation in Speech Motor Control, John F. Houde and Michael I. Jordan . . . . 38

Learning Human-like Ifriowledge by Singular Value Decomposition: A Progress

Report, Thomas K. Landauer, Darrell Laham and Peter Foltz 45

Mülti-modular Associative Memory, NirLevy, David Hörn and Eytan Ruppin . . . 52

Serial Order in Reading Aloud: Connectionist Models and Neighborhood Structure, Jeanne C. Milostan and Garrison W. Cottrell 59 A Superadditive-Impairment Theory of Optic Aphasia, Michael C. Mozer, Mark Sitton and Martha Farah 66 A Hippocampal Model ofRecognition Memory, Randall C. O'Reilly, Kenneth A. Norman and James L. McClelland 73

Correlates of Attention in a Model of Dynamic Visual Recognition, Rajesh P N. Rao 80

Recurrent Neural Networks Can Learn to Implement Symbol-Sensitive Counting, Paul Rodriguez and JanetWiles . . . 87

Comparison of Human and Machine Word Recognition, Markus Schenkel, Cyril Latimer and Marwan Jabri 94

Part II Neuroscience

Coding of Naturalistic Stimuli by Auditory Midbrain Neurons, Hagai Attias and Christoph E. Schreiner . 103

vi Contents

Refractoriness and Neural Precision, Michael J. Berry II and Markus Meister . . . 110

Statistical Models of Conditioning, Peter Dayan and Theresa Long 117

Characterizing Neurons in the Primary Auditory Cortex ofthe Awake Primate Using Reverse Correlation, R. Christopher deChanns and Michael M. Merzenich . 124

Using Helmholtz Machines to Analyze Multi-channel Neuronal Recordings, Virginia R. de Sa, R. Christopher deChanns and Michael M. Merzenich 131

Instabilities in Eye Movement Control: A Model of Periodic Alternating Nystagmus, Ernst R. Dow and Thomas J. Anastasio 138

Hippocampal Model ofRat Spatial Abilities Using Temporal Difference

Learning, David J. Foster, Richard G. M. Morris and Peter Dayan 145

GradientsforRetinotectalMapping,Geof(RyJ.Goodtä\l 152

A Mathematical Model ofAxon Guidance by Diffusible Factors, Geoffrey J. Goodhill 159 Computing withAction Potentials (Invited Talk), John J. Hopfield, Carlos D. Brody and Sam Roweis 166

A Model ofEarly Visual Processing, Laurent M, Jochen Braun, Dale K. Lee and Christof Koch 173

Perturbative M-Sequences for Auditory Systems Identification, Mark Kvale and Christoph E. Schreiner 180

Effects of Spike Urning Underlying Binocular Integration andRivalry in a Neural Model ofEarly Visual Cortex, Erik D. Lumer 187

Dynamic Stochastic Synapses as Computational Units, Wolfgang Maass and Anthony M. Zador 194

Synaptic Transmission: An Information-Theoretic Perspective, Amit Manwani and Christof Koch 201

Toward a Single-Cell Account for Binocular Disparity Tuning: An Energy Model May Be Hiding in Your Dendrites, BartlettW. Mel, Daniel L. Ruderman and Kevin A. Archie 208

Just One View: Invariances in Inferotemporal Cell Tuning, Maximilian Riesenhuber and Tomaso Poggio 215

On the Separation of Signals from Neighboring Cells in Tetrode Recordings, Maneesh Sahani, John S. Pezaris and Richard A. Andersen 222

Independent Component Analysis for Identification ofArtifacts in Magnetoencephalographic Recordings, Ricardo Vigärio, Veikko Jousmäki, Matti Hämäläinen, Riitta Hari and Erkki Oja . 229

Modeling Complex Cells in an Awake Macaque during Natural Image Viewing, William E.Vinje and Jack L. Gallant 236

%

Contents vii

Part i n Theory

The Canonical Distortion Measure in Feature Space and 1-NN Classification,

Jonathan Baxter and Peter Bartlett 245

Multiple Threshold Neural Logic, Vasken Bohossian and Jehoshua Brück 252

Generalization in Decision Trees andDNF: Does Size Matter?

Mostefa Golea, Peter Bartlett, Wee Sun Lee and LlewMason 259

Selecting Weighting Factors in Logarithmic Opinion Pools, Tom Heskes 266

New Approximations of Differential Entropyfor Independent Component Analysis and Projection Pursuit, Aapo Hyvärinen 273 Boltzmann Machine Learning Using Mean Field Theory and Linear Response Correction, Hilbert J. Kappen and F. B. Rodriguez 280 Relative Loss Boundsfor Multidimensional Regression Problems, Jyrki Kivinen and Manfred K. Warmuth 287

Asymptotic Theory for Regularization: One-Dimensional Linear Case, Petri Koistinen 294

Two Approaches to Optimal Armealing, ToddK. Leen, Bernhard Schottky and David Saad 301

Structural Risk Minimizationfor Nonparametric Time Series Prediction, RonMeir f: 308

Analytical Study ofthe Interplay between Architecture and Predictability,

Avner Priel, Ido Kanter and David A. Kessler 315

Globally Optimal On-line Learning Rules, Magnus Rattray and David Saad . . . . 322

Minimax and Hamiltonian Dynamics of Excitatory-Inhibitory Networks, H. Sebastian Seung, Tom J. Richardson, Jeffrey C. Lagarias and John J. Hopfield . 329 Data-Dependent Structural Risk Minimizationfor Perceptron Decision Trees, John Shawe-Taylor and Nello Cristianini 336

From Regularization Operators to Support Vector Kernels, Alex J. Smola and Bernhard Schölkopf 343

The Rectified Gaussian Distribution, Nicholas D. Socci, Daniel D. Lee and H. Sebastian Seung 350

On-line Learning from Finite Training Sets in Nonlinear Networks,

Peter Sollich and David Barber 357

Competitive On-line Linear Regression, VolodyaVovk 364

On the Infeasibility of Training Neural Networks with Small Squared Errors, VanH.Vu 371 The Storage Capacity ofa Fully-Connected Committee Machine, Yuansheng Xiong, Chulan Kwon and Jong-Hoon Oh 378

vii'i Contents

The Efficiency and the Robustness of Natural Gradient Descent Learning Rule, Howard H. Yang and Shun-ichi Amari 385

Part IV Algorithms and Architecture

Ensemble Learning for Multi-Layer Networks, David Barberand Christopher M. Bishop 395

Radial Basis Functions: A Bayesian Treatment, David Barber and Bernhard Schottky 402

Shared Context Probabilistic Transducers, Yoshua Bengio, Samy Bengio, Jean-Francois Isabelle and Yoram Singer 409

Approximating Posterior Distributions in Belief Networks Using Mixtures, Christopher M. Bishop, Neil Lawrence, Tommi Jaakkola and Michael I. Jordan . . 416

Receptive Field Formation in Natural Scene Environments: Comparison of Single Cell Learning Rules, BrianS. Blais, Nathan Intrator, Harel Shouval and Leon N. Cooper 423

An Annealed Self-Organizing Map for Source Channel Coding, Matthias Burger, Thore Graepel and Klaus Obermayer 430

Incorporating Test Inputs into Learning, Zehra Cataltepe and Malik Magdon-Ismail 437

On Efficient Heuristic Ranking ofHypotheses, Steve Chien, Andre Stechert and Darren Mutz 444

Learning to Order Things, William W. Cohen, Robert E. Schapire and Yoram Singer 451

Regularisation in Sequential Learning Algorithms, Joäo F. G. de Freitas, Mahesan Niranjan and Andrew H. Gee 458

Agnostic Classification ofMarkovian Sequences, RanEl-Yaniv, ShaiFineandNaftaliTishby 465

Ensemble and Modular Approaches for Face Detection: A Comparison, Raphael Feraud and Olivier Bernier . 472

A Revolution: Belief Propagation in Graphs with Cycles, Brendan J. Frey and David J. C. MacKay 479

Hierarchical Non-linear Factor Analysis and Topographie Mops, Zoubin Ghahramani and Geoffrey E. Hinton 486

Regression with Input-dependent Noise: A Gaussian Process Treatment, Paul W. Goldberg, Christopher K. I. Williams and Christopher M. Bishop 493

Linear Concepts and Hidden variables: An Empirical Study, Adam J. Grove and Dan Roth 500

Classification by Pairwise Coupling, Trevor Hastie and Robert Tibshirani 507

Contents S /' üc

Unsupervised On-line Learning ofDecision Treesfor Hierarchical Data Analysis, Marcus Held and Joachim M. Buhmann 514

Nonlinear Markov Networks for ContinuoUs Variables,

Reimar Hof mann and Volker Tresp 521

Active Data Clustering, Thomas Hofmann and Joachim M. Buhmann 528

Function Approximation with the Sweeping Hinge Algorithm, Don R. Hush, Fernando Lozano and Bill Home 535 The Error Coding and Substitution PaCTs, Gareüi James and Trevor Hastie . . . . 542

S-Map: A Network with a Simple Self-Organization Algorithmfor Generative Topographie Mappings, Kimmo Kiviluoto and Erkki Oja . 549

Learning Nonlinear Overcomplete Representations for Efficient Coding,

Michaels.LewickiandTerrenceJ. Sejnowski 556

Factorizing Multivariate Function Classes, Juan K. Lin 563

A Framework for Multiple-Instance Learning, Oded Maron and Tomas Lozano-Perez 570 An Application of Reversible-Jump MCMC to Multivariate Spherical Gaussian Mixtures, Alan D. Marrs 577

Estimating Dependency Structure as a Hidden Variable,

Marina Meilä and Michael I. Jordan 584

Combining Classifiers Using Correspondence Analysis, Christopher J. Merz . . . 591

Learning Path Distributions Using Nonequilibrium Diffusion Networks, Paul Mineiro, Javier Movellan and Ruth J. Williams 598 Learning Generative Models with the Up-Propagation Algorithm, Jong-Hoon Oh and H. Sebastian Seung 605

An Incremental Nearest Neighbor Algorithm with Queries, JoelRatsaby 612

RCC Cannot Compute Certain FSA, Even with Arbitrary Transfer Functions, MarkRing 619

EMAlgorithmsfor PCA and SPCA, Sam Roweis 626

Locol Dimensionality Reduction, Stefan Schaal, Sethu Vijayakumar and Christopher G. Atkeson 633

Prior Knowledge in Support Vector Kernels, Bernhard Schölkopf, Patrice Simard, Alex J. Smola and Vladimir Vapnik 640

Training Methods for Adaptive Boosting of Neural Networks,

Holger Schwenk and Yoshua Bengio 647

Learning Continuous Attractors in Recurrent Networks, H. Sebastian Seung . . . 654

Monotonie Networks, Joseph Sill 661

Stacked Density Estimation, Padhraic Smyth and David Wolpert 668

X Contents

Bidirectional Retrievalfrom Associative Memory,

Friedrich T. Sommer and Günther Palm 675

Mapping a Manifold of Perceptual Observations, Joshua B. Tenenbaum 682

Graph Matching with Hierarchical Discrete Relaxation, Richard C. Wilson and Edwin R. Hancock 689 Multiplicative Updating Rulefor Blind Separation Derivedfrom the Method of Scoring, Howard H. Yang 696

PartV Implementation

A 1,000-Neuron System with One Million 7-bit Physical Interconnections,

YuzoHirai 705

Silicon Retina with Adaptive Filtering Properties, Shih-ChiiLiu 712

Analog VLSI Model oflntersegmental Coordination with Nearest-Neighbor Coupling, Girish N. Patel, Jeremy H. Holleman and Stephen P. DeWeerth 719 An Analog VLSI Neural Network for Phase-based Machine Vision, Bertram E.Shi and KwokFai Hui 726

Part VI Speech, Handwriting and Signal Processing

Analysis ofDrifting Dynamics with Neural Network Hidden Markov Models, Jens Kohlmorgen, Klaus-Robert Müller and Klaus Pawelzik 735

Bayesian Robustification for Audio Visual Fusion, '**' Javier Movellan and Paul Mineiro 742

Modeling Acoustic Correlations by Factor Analysis, Lawrence Saul and Mazin Rahim 749

Blind Separation of Radio Signals in Fading Channels, Kari Torkkola 756

Hybrid NN/HMM-Based Speech Recognition with a Discriminant Neural Feature Extraction, Daniel Willett and Gerhard Rigoll 763

Part VII Visual Processing

A Non-Parametric Multi-Scale Statistical Model for Natural Images, Jeremy S. De Bonet and Paul A. Viola 773

Recovering Perspective Pose with a Dual Step EMAlgorithm,

Andrew D. J. Gross and Edwin R. Hancock 780

Bayesian Model ofSurface Perception, William T. Freeman and Paul A. Viola . . 787

Features as Sufficient Statistics, Davi Geiger, Archisman Rudra and Laurance T. Maloney 794 Detection of First and Second Order Motion, Alexander Grunewald and Heiko Neumann 801

Contents xi

A Simple and Fast Neu'ral Network Approachto Stereovision, Rolf D.Henkel . . . 808

Inferring Sparse, Overcomplete Image Codes Using an Efficient Coding

Framework, Michael S. Lewicki and Bruno Ä. Olshausen 815

Visual Navigation in a Robot Using Zig-Zag Behavior, M. Anthony Lewis 822

2D Observersfor Human 3D Object Recognition? Zili Liu and Daniel Kersten . . 829

Self-similarity Properties of Natural Images, Antonio Turiel, Germän Mato, Nestor Parga and Jean-Pierre Nadal 836 Multiresolution Tangent Distance for Affine-invariant Classification, Nuno Vasconcelos and Andrew Lippman 843

Phase Transitions and the Perceptual Organization of Video Sequences, YairWeiss 850

PartVIÜ Applications

Using Expectation to Guide Processing: A Study ofThree Real-World Applications, Shumeet Baluja 859

Structure Driven Image Database Retrieval, Jeremy S. De Bonet and Paul A. Viola 866

A General Purpose Image Processing Chip: Orientation Detection, Ralph Etienne-Curnnüngs and Donghui Cai 873

An Analog VLSI Model ofthe Fly Elementary Motion Detector, ReidR. Harrison and Christof Koch 880

MELONETI: Neural Nets for Inventing Baroque-Style Chorale Variations, Dominik Hörnel 887

Extended ICA Removes Artifacts from Electroencephalographic Recordings, Tzyy-Ping Jung, Colin Humphries, Te-Won Lee, Scott Makeig, Martin J. McKeown, Vicente Iragui and Terrence J. Sejnowski 894

A Generic Approachfor Identification of Event Related Brain Potentials via a Competitive Neural Network Structure, Daniel H. Lange, Hava T. Siegelmann, Hillel Pratt and Gideon F. Inbar 901

A Neural Network Based Head Tracking System,

Daniel D. Lee and H. Sebastian Seung 908

Wavelet Models for Video Time-Series, ShengMa and ChuanyiJi 915

Reinforcement Leaming for Call Admission Control and Routing in Integrated Service Networks, Peter Marbach, Oliver Mihatsch, Miriam Schulte and John N. Tsitsiklis 922 Leaming to Schedule Straight-Line Code, Eliot Moss, Paul Utgoff, John Cavazos, Doina Precup, Darko Stefanovic, Carla Brodley and David Scheeff . 929

Enhancing Q-Learning for Optimal Asset Allocation, Ralph Neuneier 936

XU Contents

Intrusion Detection with Neural Networks, Jake Ryan, Meng-Jang Lin and Risto Miikkulainen 943

Incorporating Contextual Information in White Blood Cell Identification, Xubo Song, Yaser Abu-Mostafa, Joseph Sill and Harvey Kasdan 950

Bach in a Box—Real-Time Harmony, Randall R. Spangler, Rodney M. Goodman and Jim Hawkins 957

Experiences with Bayesian Learning in a Real World Application, Peter Sykacek, Georg Dorffher, Peter Rappelsbergerand Josef Zeitlhofer 964

A Solution for Missing Data in Recurrent Neural Networks with an Application to Blood Glucose Prediction, Volker Tresp and Thomas Briegel 971

Use ofa Multi-Layer Perceptron to Predict Malignancy in Ovarian Tumors,

Herman Verreist, Yves Moreau, Joos Vandewalle and Dirk Trmmerman 978

Modelling Seasonality and Trends in Daily Rainfall Data, Peter M. Williams . . . 985

The Observer-Observation Dilemma in Neuro-Forecasting, Hans Georg Zimmermann and Ralph Neüneier 992

Part IX Control, Navigation and Planning

Generalized Prioritized Sweeping, David Andre, Nir Friedman and Ronald Parr . . 1001

Nonparametric Model-Based Reinforcement Learning, Christopher G. Atkeson . . 1008

An Improved Policy Iteration Algorithmfor Partially Observable MDPs, Eric A. Hansen > 1015

Automated Aircraft Recovery via Reinforcement Learning: Initial Experiments, Jeffrey F. Monaco, David G. Ward and Andrew G. Barto 1022

Reinforcement Learning for Continuous Stochastic Control Problems,

Remi Munos and Paul Bourgine 1029

Adaptive Choice ofGrid and Time in Reinforcement Learning, Stephan Pareigis . . 1036

Reinforcement Learning with Hierarchies of Machines, Ronald Parr and Stuart Russell 1043 Multi-time Models for Temporally Abstract Planning, Doina Precup and Richard S. Sutton 1050

How to Dynamically Merge Markov Decision Processes,

Satinder Singh and David Cohn 1057

The Asymptotic Convergence-Rate of Q-karning, Csäba. Szepesväri . . . . . . . . 1064

Hybrid Reinforcement Learning and Its Application to Biped Robot Control, Satoshi Yamada, AkiraWatanabeandMichioNakashima 1071 Index ofAuthors 1079

Keyword Index 1083