Associative Pattern Memory (APM)Larry WerthJuly 14, 2007

Introduction and Background of APM

• Human Associative Pattern Memory• Computer Implemented APM• Basis for Two Successful Startup Companies• Six Patents Granted and Others Pending• Successful Implementation of NKS

Objective of My Presentation

• Describe the APM Concept & Implementation• Describe its Advantages / Features• Identify Types of Applications• Describe its Current Status and Future Goals

Origin of Concept

• Randomly Connected Neural Network Models• States Sequence Terminates in a Cycle• Randomly Map Each State to an Input Pattern• Sampled Pattern Value & Current State

Determine Next State• The Ultimate Cycle Represents the Input

Pattern• Cycles Form the Basis of the APM

Cycle PropertiesRandomly Connected DFA’s

Expected# Expected# Expected#Fraction

Terminal Number TransitionTerminal

Total States (N) States(S) Cycles(C) States(T) States(F)100 12 3 7 .121,000 40 4 20 .04010,000 125 5 63

.0125100,000 396 6 198

.003961,000,000 1,253 8 627

.00125310,000,000 3,963 9 1982

.000396100,000,000 12,533 10 6267

.00012531,000,000,000 39,632 11 19817 .0000396

Conceptual Implementation of APM

State Array

Next State Array (Value = 0)

Next State Array (Value = 1)

Input Pattern Array

Pattern Address

Pattern Value

Next State Address

CurrentState Address

Response Array

Respond to Pattern(Read From Cycle Addresses)

Train Pattern(Write to Cycle Addresses)

State Array: Filled with Random Pattern AddressesNext State Arrays : Filled with Random State AddressesResponse Array: Assigned Responses to Patterns

Solution to Multiple Cycles

• Introduce a Refractory Period• A State Can Not Occur Again Until After a

Specified Number of Steps• Establishes a Minimum Cycle Length• Assures One Cycle Per Input Pattern

Independent of Initial State• Input Pattern is Represented By a Single

Sequence of Random Addresses in Memory

Minimum Cycle Length Example

• Number of States: 1,000,000• Minimum Cycle Length: 3,700• Probability of a Second Cycle of 3,700 in

Length: 1 in 1,000,000

Based on the probability of not picking one of 3700 in 1,000,000 after 3700 tries.

Response/Recognition Capacity

• During Training Desired Responses are Written to Cycle Addresses in Response Memory

• Problem: Response Memory Fills UP Quickly• Any Cycle Address has Memory of Previous

Input Sample Values• Do Not Need to Use All Cycle Addresses• Solution: Vertical Sensors

Vertical Sensor Cycle Detection

State Addresses Plane With Cycle

Vertical Sensors Detect Presence Absence of Cycle

Upper Memory Plane forms New Input Pattern Based on Sensor Status

Vertical Sensor Implementation

• Number of States: 1,000,000• Minimum Cycle Length: 3,700• One of 270 Addresses are in Cycles• Vertical Sensor Field Size: 135• Probability Field Contains Cycle Address: .5• Vertical Sensor Determines Bit Status of Hash

Values that Addresses Response Memory

Fuzzy Hash

• Similar Input Patterns Produce Similar Cycles• Similar Input Patterns Generate the same or

Similar Hash Codes• Multiple Independent Hash Codes are

Generated By One Cycle (One Input Pattern)• A Voting Mode For Response Identification

Contributes to Fuzzy Recognition

Advantages of Using Cycles

• Creates a Fuzzy Hash• Simple and Fast Implementation• Common Language for Different Pattern Types • Spatial and Temporal Integration to Form New

Higher Level Input Patterns• Automatic Segmentation of Time Varying

Patterns

Applications

• Actual Applications: Hand Printed Character Recognition, Machine Vision, Video Compression, Financial Pattern Forecasting

• Signal Processing – Vector Quantization• Video Surveillance – Smart Cameras• Video Object Tracking• Stereo Vision

Current Status and Objectives

• Software Library Written in C/C++• Objective: General Purpose Tool for Pattern

Recognition Development• Looking for a Business Partner• Software Will be Available on Our Web Site

www.netwerth.net