leonid perlovsky visiting scholar, harvard university technical advisor, afrl

BIO-INSPIRED AND COGNITIVE COMPUTINGfor

data mining, tracking, fusion, financial prediction, language understanding, web search engines, and

diagnostic modeling of cultures

Leonid PerlovskyVisiting Scholar, Harvard University

Technical Advisor, AFRL

IEEE 2007 Fall Short Course

Holiday Inn, Woburn MA

7:00 – 9:00 pm, Nov. 8, 15, 22, Dec. 6

OUTLINE

1. Cognition, Complexity, and Logic

2. The Knowledge Instinct -Neural Modeling Fields and Dynamic Logic

3. Language

4. Integration of cognition and language

5. High Cognitive Functions

6. Evolution of cultures

7. Future directions

DETAILED OUTLINE

1. Cognition – integration of real-time signals and a priori knowledge

1.1. physics and mathematics of the mind1.2. genetic argument for the first principles1.3. the nature of understanding1.3.1. concepts: “chair”1.3.2. hierarchy1.4. combinatorial complexity (CC) – a fundamental

problem?1.5. CC since 1950s1.6. CC vs. logic1.6.1. formal, multivalued and fuzzy logics1.6.2. dynamic logic1.6.3. Aristotle vs. Godel + Alexander the Great1.7. mathematics vs. mind1.8. structure of the mind: concepts, instincts, emotions,

behavior1.9. the knowledge instinct1.9.1. need for learning1.9.2. “knowledge emotion” = aesthetic emotion

2. Modeling Field Theory (NMF) of cognition2.1. the knowledge instinct = max similarity2.2. Similarity as likelihood, as information2.3. Dynamic Logic (DL)

2.4. applications, examples, exercises 2.4.1 clustering2.4.2 tracking and CRB

- example of tracking below clutter- complexity NMF vs. MHT- models

2.4.3 recognition- example of pattern in image below clutter- complexity NMF vs. MHT- models

2.4.4 fusion- example of fusion, navigation, and detection below clutter- models

2.4.5 prediction- financial prediction- models

2.5. block-diagrams2.7. hierarchical structure

3. Language3.1. language acquisition and complexity3.1.2. language separate from cognition3.1.3. hierarchy of language3.1.4. application

- search engine based on understanding

3.2. NMF of language

3.2.1 differentiating non-differentiable qualitative functions

DETAILED OUTLINECONTINUATION

4. Integration of cognition and language4.1. language vs. cognition4.2. past: AI and Chomskyan linguistics4.3. integrated models4.3. integrated hierarchies4.4. Humboldt’s inner linguistic form

5. Prolegomena to a theory of the mind5.1. higher cognitive functions5.2. from Plato to Lock5.3. from Kant to Grossberg5.4. NMF vs. Buddhism5.5. NMF vs. neuro-biology5.5. NMF dynamics: elementary thought process5.6. consciousness and unconscious5.7. aesthetics and beauty5.8. intuition5.9. why Adam was expelled from paradise5.9. symbols and signs 5.9.1. NMF theory of symbols5.9.2. symbolic AI: “bewitchment”5.10. why Adam was expelled from paradise

6. Evolution of Culture 6.1. Culture and language6.2. KI: differentiation and synthesis6.3. “Spiritual” cultural measurements6.4. Mathematical modeling and

predictions6.4.1. dynamic culture6.4.2. traditional culture6.4.3. terrorist’s consciousness6.4.4. interacting cultures6.5. Evolution of concepts and emotions6.6. Creativity6.7. Disintegration of cultures 6.8. Emotions in language6.9. English vs. Arabic 6.10. Synthesis 6.11. Differentiation of emotions6.12. Role of music in evolution of the

mind and culture

7. Future directions & publications

7.1. Science and Religion7.2 Predictions and testing7.3. Future directions7.4 Publications

INTRODUCTION

Nature of the mind

PHYSICS AND MATHEMATICS OF THE MINDRANGE OF CONCEPTS

Logic is sufficient to explain mind– [Newell, “Artificial Intelligence”, 1980s]

No new specific mathematical concepts are needed– Mind is a collection of ad-hoc principles, [Minsky, 1990s]

Specific mathematical constructs describe the multiplicity of mind phenomena– “first physical principles of mind”– [Grossberg, Zadeh, Perlovsky,…]

Quantum computation– [Hameroff, Penrose, Perlovsky,…]

New unknown yet physical phenomena– [Josephson, Penrose]

GENETIC ARGUMENTSFOR THE “FIRST PRINCIPLES”

Only 30,000 genes in human genome– Only about 2% difference between human and apes– Say, 1% difference between human and ape minds– Only about 300 proteins

Therefore, the mind has to utilize few inborn principles – If we count “a protein per concept”– If we count combinations: 300300 ~ unlimited => all concepts

and languages could have been genetically h/w-ed (!?!)

Languages and concepts are not genetically hardwired– Because they have to be flexible and adaptive

COGNITION

• Understanding the world around– Perception– Simple objects– Complex situations

• Integration of real-time signals and existing (a priori) knowledge – From signals to concepts– From less knowledge to more knowledge

EXAMPLE

Example: “this is a chair, it is for sitting”

Identify objects– signals -> concepts

What in the mind help us do this? Representations, models, ontologies?– What is the nature of representations in the

mind?– Wooden chairs in the world, but no wood in the

brain

VISUAL PERCEPTION

Neural mechanisms are well studied– Projection from retina to visual cortex (geometrically accurate) – Projection of memories-models

• from memory to visual cortex– Matching: sensory signals and models– In visual nerve more feedback connections than feedforward

• matching involves complicated adaptation of models and signals

Difficulty– Associate signals with models– A lot of models (expected objects and scences)– Many more combinations: models<->pixels

Association + adaptation– To adapt, signals and models should be associated– To associate, they should be adapted

ALGORITHMIC DIFFICULTIES A FUNDAMENTAL PROBLEM?

Cognition and language involve evaluating large numbers of combinations– Pixels -> objects -> scenes

Combinatorial Complexity (CC) – A general problem (since the 1950s)

• Detection, recognition, tracking, fusion, situational awareness, language…

• Pattern recognition, neural networks, rule systems…

Combinations of 100 elements are 100100

– This number ~ the size of the Universe • > all the events in the Universe during its entire life

CC was encountered for over 50 years

Statistical pattern recognition and neural networks: CC of learning requirements

Rule systems and AI, in the presence of variability : CC of rules

– Minsky 1960s: Artificial Intelligence– Chomsky 1957: language mechanisms are rule systems

Model-based systems, with adaptive models: CC of computations

– Chomsky 1981: language mechanisms are model-based (rules and parameters)

Current ontologies, “semantic web” are rule-systems– Evolvable ontologies : present challenge

COMBINATORIAL COMPLEXITY SINCE the 1950s

CC AND TYPES OF LOGIC

CC is related to formal logic– Law of excluded middle (or excluded third)

•every logical statement is either true or false– Gödel proved that logic is “illogical,” “inconsistent” (1930s)

– CC is Gödel's “incompleteness” in a finite system

Multivalued logic eliminated the “law of excluded third”– Still, the math. of formal logic– Excluded 3rd -> excluded (n+1)

Fuzzy logic eliminated the “law of excluded third”– Fuzzy logic systems are either too fuzzy or too crisp– The mind fits fuzziness for every statement at every step => CC

Logic pervades all algorithms and neural networks – rule systems, fuzzy systems (degree of fuzziness), pattern recognition, neural networks (training uses logical statements)

LOGIC VS. GRADIENT ASCENT

Gradient ascent maximizes without CC– Requires continuous parameters– How to take gradients along “association”?

• Data Xn (or) to object m• It is a logical statement, discrete, non-differentiable

– Models / ontologies require logic => CC

Multivalued logic does not lead to gradient ascent

Fuzzy logic uses continuous association variables, but no parameters to differentiate– A new principle is needed to specify gradient ascent along

fuzzy associations: dynamic logic

DYNAMIC LOGIC

Dynamic Logic unifies formal and fuzzy logic– initial “vague or fuzzy concepts” dynamically

evolve into “formal-logic or crisp concepts”

Dynamic logic– based on a similarity between models and

signals

Overcomes CC of model-based recognition – fast algorithms

ARISTOTLE VS. GÖDEL logic, forms, and language

Aristotle– Logic: a supreme way of argument– Forms: representations in the mind

Form-as-potentiality evolves into form-as-actuality Logic is valid for actualities, not for potentialities (Dynamic Logic)

– Thought language and thinking are closely linked Language contains the necessary uncertainty

From Boole to Russell: formalization of logic– Logicians eliminated from logic uncertainty of language– Hilbert: formalize rules of mathematical proofs forever

Gödel (the 1930s) – Logic is not consistent

Any statement can be proved true and false

Aristotle and Alexander the Great

OUTLINE

• Cognition, complexity, and logic- Logic does not work, but the mind does

• The Mind and Knowledge Instinct- Neural Modeling Fields and Dynamic Logic

• Language

• Integration of cognition and language

• Higher Cognitive Functions

• Future directions

STRUCTURE OF THE MIND

Concepts – Models of objects, their relations, and situations– Evolved to satisfy instincts

Instincts– Internal sensors (e.g. sugar level in blood)

Emotions– Neural signals connecting instincts and concepts

• e.g. a hungry person sees food all around

Behavior– Models of goals (desires) and muscle-movement…

Hierarchy– Concept-models and behavior-models are organized in a “loose”

hierarchy

THE KNOWLEDGE INSTINCT

Model-concepts always have to be adapted– lighting, surrounding, new objects and situations

– even when there is no concrete “bodily” needs

Instinct for knowledge and understanding– Increase similarity between models and the world

Emotions related to the knowledge instinct– Satisfaction or dissatisfaction

• change in similarity between models and world

– Related not to bodily instincts• harmony or disharmony (knowledge-world): aesthetic emotion

REASONS FOR PAST LIMITATIONS

Human intelligence combines conceptual understanding with emotional evaluation

A long-standing cultural belief that emotions are opposite to thinking and intellect– “Stay cool to be smart”– Socrates, Plato, Aristotle– Reiterated by founders of Artificial Intelligence [Newell,

Minsky]

Neural Modeling Fields (NMF)

A mathematical construct modeling the mind– Neural synaptic fields represent model-concepts– A loose hierarchy of more and more general concepts– At every level: bottom-up signals, top-down signals– At every level: concepts, emotions, models, behavior– Concepts become input signals to the next level

NEURAL MODELING FIELDSbasic two-layer mechanism: from signals to

concepts

Signals– Pixels or samples (from sensor or retina)

x(n), n = 1,…,N

Concept-Models (objects or situations) Mm(Sm,n), parameters Sm, m = 1, …;

– Models predict expected signals from objects

Goal: learn object-models and understand signals (knowledge instinct)

THE KNOWLEDGE INSTINCT

The knowledge instinct = maximization of similarity between signals and models

Similarity between signals and models, L– L = l ({x}) = l (x(n))

– l (x(n)) = r(m) l (x(n) | Mm(Sm,n))

– l (x(n) | Mm(Sm,n)) is a conditional similarity for x(n) given m

• {n} are not independent, M(n) may depend on n’

CC: L contains MN items: all associations of pixels and models (LOGIC)

n

m

SIMILARITY

Similarity as likelihood

– l (x(n) | Mm(Sm,n)) = pdf(x(n) | Mm(Sm,n)),– a conditional pdf for x(n) given m– e.g., Gaussian pdf(X(n)|m) = G(X(n)|Mm,Cm) = 2-d/2 detCm

-1/2 exp(-DmnTCm

-1 Dmn/2); Dmn = X(n) – Mm(n)

– Note, this is NOT the usual “Gaussian assumption” • deviations from models D are random, not the data X• multiple models {m} can model any pdf, not one Gaussian model

– Use for sets of data points

Similarity as information

– l (x(n) | Mm(Sm,n)) = abs(x(n))*pdf(x(n) | Mm(Sm,n)),– a mutual information in model m on data x(n)– L is a mutual information in all model about all data– e.g., Gaussian pdf(X(n)|m) = G(X(n)|Mm,Cm)

– Use for continuous data (signals, images)

DYNAMIC LOGIC (DL) non-combinatorial solution

Start with a set of signals and unknown object-models– any parameter values Sm

– associate object-model with its contents (signal composition)

– (1) f(m|n) = r(m) l (n|m) / r(m') l (n|m')

Improve parameter estimation– (2) Sm = Sm + f(m|n) [ln l (n|m)/Mm]*[Mm/Sm]

• ( determines speed of convergence)

– learn signal-contents of objects

Continue iterations (1)-(2). Theorem: MF is a converging system

- similarity increases on each iteration- aesthetic emotion is positive during learning

'm

n

OUTLINE

• Cognition, complexity, and logic- Logic does not work, but the mind does

• The Mind and Knowledge Instinct- Neural Modeling Fields and Dynamic Logic- Application examples

• Language




APPLICATIONS

Many applications have been developed– Government– Medical– Commercial (about 25 companies use this technology)

Sensor signals processing and object recognition– Variety of sensors

Financial market predictions– Market crash on 9/11 predicted a week ahead

Internet search engines– Based on text understanding

Evolving ontologies for Semantic Web

Every application needs models– Future self-evolving models: integrated cognition and language

APPLICATION 1 - CLUSTERING

Find “natural” groups or clusters in data Use Gaussian pdf and simple models

l (n|m) = 2-d/2 detCm-1/2 exp(-Dmn

TCm-1 Dmn/2); Dmn = X(n) – Mm(n)

Mm(n) = Mm; each model has just 1 parameter, Sm = Mm

– This is clustering with Gaussian Mixture Model

For complex l(n|m) derivatives can be taken numerically For simple l(n|m) derivatives can be taken manually

– Simplification, not essential

Simplify parameter estimation equation for Gaussian pdf and simple modelsln l (n|m)/Mm = (-Dmn

TCm-1 Dmn) /Mm = Cm

-1 Dmn + DmnT

Cm-1 = 2 Cm

-1 Dmn, (C is symmetric)

Mm = Mm + f(m|n) Cm-1 Dmn

…

In this case, even simpler equations can be derived samples in class m: Nm = f(m|n); N = Nm

rates (priors): rm = Nm / N

means: Mm = f(m|n) X(n) / Nm

covariances: Cm = f(m|n) Dmn * DmnT / Nm

- simple interpretation: Nm, Mm, Cm are weighted averages. The only difference from standard mean and covariance estimation is weights f(m|n), probabilities of class m

These are iterative equations, f(m|n) depends on parameters; theorem: iterations converge

n

n

n

n

m

EXERCISE (1) – HOME WORK

Code a simple NMF/DL for Gaussian Mixture clustering

(1) Simulate data– Specify true parameters, say:

• Dimensionality, d = 2• Number of classes, 3, m = 1, 2, 3; • Rates rm= 0.2, 0.3, 0.5; • Number of samples, N = 100; Nm= N*rm= 20, 30, 50; • Means, Mm= …; 2-d vectors, (0.1, 0.1), (0.2, 0.8) (0.8, 0.2) • Covariances, Cm = unit matrixes

– Call a function that generates Gaussian data

(2) Run NMF/DL code to estimate parameters (rm, Mm, Cm) and association probabilities f(m|n)

– Initiate parameters to any values, say each r = 1/3; means should not be the same, say M = (0,0), (0,1), (1,1); covariances should be initiated to larger values than uncertainties in the means (squared), say each C = diag(2,2).

– Run iterations: estimate f(m|n), estimate parameters… until, say (changes in M) < 0.01

(3) Plot results (2-d plots)– Plot data for each class in different color/symbols– Plot means– Plot covariances

• 2- ellipses around their means: (X-M)TC-1(X-M) = 2

APPLICATION 2 - TRACKING

1) Example

2) Complexity of computations

3) Exercise

4) Cramer-Rao Bounds

Example 2: GMTI Tracking and Detection

Below Clutter

18 dB improvement

0 1 kmCross-Range

Ra

ng

e1

km0

(a)True

Tracks

b

Ra

ng

e1

km0

Initial state of model 2 iterations

5 iterations 9 iterations 12 iterations Converged state

DL starts with uncertain knowledge and converges rapidly on exact solution

EXAMPLE (2) note page

Detection and tracking targets below clutter: (a) true track positions in 0.5km x 0.5km data set; (b) actual data available for detection and tracking (signal is below clutter, signal-to-clutter ratio is about –2dB for amplitude and –3dB for Doppler; 6 scans are shown on top of each other, each contains 500 data points). Dynamic logic operation: (c) an initial fuzzy model, the fuzziness corresponds to the uncertainty of knowledge; (d) to (h) show increasingly improved models at various iterations (total of 20 iterations). Between (c) and (d) the algorithm fits the data with one model, uncertainty is somewhat reduced. There are two types of models: one uniform model describing clutter (it is not shown), and linear track models with large uncertainty; the number of track models, locations, and velocities are estimated from the data. Between (d) and (e) the algorithm tried to fit the data with more than one track-model and decided, that it needs two models to ‘understand’ the content of the data. Fitting with 2 tracks continues till (f); between (f) and (g) a third track is added. Iterations stopped at (h), when similarity stopped increasing. Detected tracks closely correspond to the truth (a). Complexity of this solution is low, about 106 operations. Solving this problem by MHT (template matching with evaluating combinations of various associations – this is a standard state-of-the-art) would take about MN = 101700 operations, unsolvable.

TRACKING AND DETECTION BELOW CLUTTER (movie, same as above)

DL starts with uncertain knowledge, and similar to human mind does not sort through all possibilities, but converges rapidly on exact solution

3 targets, 6 scans, signal-to-clutter, S/C ~ -3.0dB

TRACKING EXAMPLEcomplexity and improvement

•Technical difficulty-Signal/Clutter = - 3 dB, standard tracking requirements

15 dB-Computations, standard hypothesis testing ~ 101700,

unsolvable

• Solved by Dynamic Logic-Computations: 2x107

-Improvement 18 dB

EXERCISE (2)

Develop NMF/DL for target trackingStep 1 (and the only one)

– Develop models for tracking– Model: where do you expect to see the target– Parameters, Sm: position xm and velocity vm

– Mm(xm, vm , n) = xm + vm * tn

– Time, tn is not a parameter of the model, but data– More complex: Keplerian trajectories– Note: typical data are 2-d () or 3-d (R,)

• Models are 3-d (or 2-d, if sufficient)• 3-d models might be “not estimatable” from 2-d data

– linear track in (x,y,z) over a short time appears linear in () – then, use 2-d models, or additional data, or longer observation

period

CRAMER-RAO BOUND (CRB)

Can a particular set of models be estimated from a particular (limited) set of data?– The question is not trivial

• A simple rule-of-thumb: N(data points) > 10*S(parameters)• In addition: use your mind: is there enough information in the data?

CRB: minimal estimation error (best possible estimation) for any algorithm or neural neworks, or… – When there are many data points, CRB is a good measure

(=ML=NMF)– When there are few data points (e.g. financial prediction) it might be

difficult to access performance• Actual errors >> CRB

Simple well-known CRB for averaging several measurementsst.dev(n) = st.dev(1)/√n

Complex CRB for tracking:– Perlovsky, L.I. (1997a). Cramer-Rao Bound for Tracking in Clutter and Tracking

Multiple Objects. Pattern Recognition Letters, 18(3), pp.283-288.

EXERCISE (2 cont.) HOMEWORK

Homework: code NMF/DL for tracking, simulate data, run the code, and plot results– When simulating data: add sensor error and clutter

• track + sensor errors: X(n,m) = xm + vm * tn + mn

• sensor error: use sensor error model (or simple Gaussian) for m,n

• clutter: X(n,m+1) = c,n ; use clutter model for c,n, or a simple uniform or Gaussian; simulate required number of clutter data points

– Do not forget to add clutter model to your NMF/FL code• l (n|m=clutter) = const; the only parameter, rc = expected proportion of

clutter

Note: the resulting system is “track-before-detect” or more accurately “concurrent detection and tracking”– does not require a standard procedure

• (1) detect, (2) associate, (3) track– association and detection is obtained together with tracking– DL requires no combinatorial searches, which often limits “track-

before-detect” performance

APPLICATION 3

FINDING PATTERNS IN IMAGES

IMAGE PATTERN BELOW NOISE

Object Image

y

Object Image + Clutter

y

x x

Multiple Hypothesis Testing (MHT) approach:try all possible ways of fitting model to the data

PRIOR STATE-OF-THE-ARTComputational complexity

For a 100 x 100 pixel image:

Number of Objects Number of Computations

1 1010

2 1020

3 1030

NMF MODELS

Information similarity measure lnl (x(n) | Mm(Sm,n)) = abs(x(n))*ln pdf(x(n) | Mm(Sm, n))

n = (nx,ny)

Clutter concept-model (m=1) pdf(X(n)|1) = r1

Object concept-model (m=2… ) pdf(x(n) | Mm(Sm, n)) = r2 G(X(n)|Mm (n,k),Cm)

Mm (n,k) = n0 + a*(k2,k); (note: k, K require no estimation)

2/

2/

Kk

Kk

ONE PATTERN BELOW CLUTTER

Y

X

SNR = -2.0 dB

DYNAMIC LOGIC WORKING

y (m

) R

ange

x (m) Cross-range

DL starts with uncertain knowledge, and similar to human mind converges rapidly on exact solution

• Object invisible to human eye

• By integrating data with the knowledge-model DL finds an object below noise

Three objects in noise object 1 object 2 object 3 SCR - 0.70 dB -1.98 dB -0.73 dB

MULTIPLE PATTERNS BELOW CLUTTER

y y

x x

3 Object Image + Clutter3 Object Image

a b c d

fe hg

IMAGE PATTERNS BELOW CLUTTER (dynamic logic iterations see note-text)

a b c d

fe hg

IMAGE PATTERNS BELOW CLUTTER (dynamic logic iterations see note-

text)

Logical complexity = MN = 105000, unsolvable; DL complexity = 107

S/C improvement ~ 16 dB

APPLICATION (3) note page

‘smile’ and ‘frown’ patterns: (a) true ‘smile’ and ‘frown’ patterns shown without clutter; (b) actual image available for recognition (signal is below clutter, signal-to-clutter ratio is between –2dB and –0.7dB); (c) an initial fuzzy model, the fuzziness corresponds to the uncertainty of knowledge; (d) to (h) show increasingly improved models at various iterations (total of 22 iterations). Between (d) and (e) the algorithm tried to fit the data with more than one model and decided, that it needs three models to ‘understand’ the content of the data. There are several types of models: one uniform model describing the clutter (it is not shown), and a variable number of blob models and parabolic models, which number, locations, and curvatures are estimated from the data. Until about (g) the algorithm ‘thought’ in terms of simple blob models, at (g) and beyond, the algorithm decided that it needs more complex parabolic models to describe the data. Iterations stopped at (h), when similarity stopped increasing. Complexity of this solution is moderate, about 1010 operations. Solving this problem by template matching would take a prohibitive 1030 to 1040 operations. (This example is discussed in more details in [[i]].) [i] Linnehan, R., Mutz, Perlovsky, L.I., C., Weijers, B., Schindler, J., Brockett, R. (2003). Detection of Patterns Below Clutter in Images. Int. Conf. On Integration of Knowledge Intensive Multi-Agent Systems, Cambridge, MA Oct.1-3, 2003.

MULTIPLE TARGET DETECTIONDL WORKING EXAMPLE

x

yDL starts with uncertain knowledge, and similar to human mind does not sort through

all possibilities like an MHT, but converges rapidly on exact solution

COMPUTATIONAL REQUIREMENTS COMPARED

Dynamic Logic (DL) vs. Classical State-of-the-art Multiple Hypothesis Testing (MHT)

Based on 100 x 100 pixel image

108 vs. 1010

2x108 vs. 1020

3x108 vs. 1030

Number of Objects

Number of Computations

DL vs. MHT

1

2

3

• Previously un-computable (1030), can now be computed (3x108 )

• This pertains to many complex information-finding problems

APPLICATION 4

SENSOR FUSIONConcurrent fusion, navigation, and detection

below clutter

SENSOR FUSION

The difficult part of sensor fusion is association of data among sensors– Which sample in one sensor corresponds to which sample in

another sensor?

If objects can be detected in each sensor individually– Still the problem of data association remains– Sometimes it is solved through coordinate estimation

• If 3-d coordinates can be estimated reliably in each sensor– Sometimes it is solved through tracking

• If objects could be reliably tracked in each sensor, => 3-d coordinates

If objects cannot be detected in each sensor individually– We have to find the best possible association among multiple

samples– This is most difficult: concurrent detection, tacking, and fusion

NMF/DL SENSOR FUSION

NMF/DL for sensor fusion requires no new conceptual development

Multiple sensor data require multiple sensor models– Data: n -> (s,n); X(n) -> X(s,n)– Models Mm(n) -> Mm(s,n)

PDF(n|m) is a product over sensors– This is a standard probabilistic procedure, another

sensor is like another dimension– pdf(m|n) -> pdf(m|s,n)

Note: this solves the difficult problem of concurrent detection, tracking, and fusion

s

Source: UAS Roadmap 2005-

2030

UNCLASSIFIED

CONCURRENT NAVIGATION, FUSION, AND DETECTION

multiple target detection and localization based on data from multiple micro-UAVs

A complex case–detection requires fusion (cannot be done with one sensor)–fusion requires exact target position estimation in 3-D–target position can be estimated by triangulation from multiple views–this requires exact UAV position

• GPS is not sufficient• UAV position - by triangulation relative to known targets

–therefore target detection and localization is performed concurrently with UAV navigation and localization, and fusion of information from multiple UAVs

Unsolvable using standard methods. Dynamic logic can solve because computational complexity scales linearly with number of sensors and targets

GEOMETRY: MULTIPLE TARGETS, MULTIPLE UAVS

UAV 1

X1 = X01 + V1t

X1=(X1,Y1,Z1)

UAV m

Xm=(Xm,Ym,Zm)

Xm = X0m + Vmt

CONDITIONAL SIMILARITIES (pdf) FOR TARGET k

Data from UAV m, sample number n, where βnm = signature position and fnm = classification feature vector:

Similarity for the data, given target k:

wheresignature position

classification features

Note: Also have a pdf for a single clutter component pdf(wnm| k=0) which is uniform in βnm, Gaussian in fnm.

Compute parameters that maximize the log-likelihood

Data Model and Likelihood Similarity

Total pdf of data samples is the summation of conditional pdfs (summation over targets plus clutter)

(mixture model)

UAV parameters target parameters classification feature parameters

Concurrent Parameter Estimation / Signature Association (NMF iterations)

Note1: bracket notation

Note2: proven to converge (e.g. EM algorithm)

Note 3: Minimum MSE solution incorporates GPS measurements

FIND SOLUTION FOR SET OF “BEST” PARAMETERS BY ITERATING BETWEEN…

Parameter Estimation and Association Probability Estimation (Bayes rule)

(probability that sample wnm was generated by target k)

Sensor 1 (of 3): Models Evolve to Locate Target Tracks in Image Data

NAVIGATION, FUSION, TRACKING, AND DETECTION (this is the basis for the previous 3 figures, all fused in x,y,z, coordinates;

double-click on the blob to play movie)

Model Parameters Iteratively Adapt to Locate the Targets

Error vs. iteration# (4 targets)

Estimated Target Position vs. iteration# (4 targets)

Parameter Estimation Errors Decrease with Increasing Number of UAVs in the Swarm

Error falls off as ~ 1/√M, where M = # UAVs in the swarm

(Note: Results are based upon Monte Carlo simulations with synthetic data)

Error in Parameter Estimates vs. clutter level and # of UAVs in the swarm

Target position UAV position

APPLICATION 5

DETECTION IN SEQUENCES OF IMAGES

DETECTION IN A SEQUENCE OF IMAGES

Signature + low noise level (SNR= 25dB)

Signature + high noise level (SNR= -6dB)

signature is present, but is obscured by noise

DETECTION IN IMAGE SEQUENCETEN ROTATION FRAMES WERE USED

Iteration 10 Iteration 100 Iteration 400

Iteration 600Compare with

Measured Image (w/o noise)

Upon convergence of the model, important parameters are estimated, including center of rotation, which will next be used for spectrum estimation.

Four model components were used, including a uniform background component. Only one component became associated with point source.

TARGET SIGNATURE

Extracted from low noise image

Extracted from high noise image

APPLICATION 6

• Radar Imaging through walls- Inverse scattering problem - Standard radar imaging algorithms (SAR) do not work

because of multi-paths, refractions, clutter

SCENARIO

RADAR IMAGING THROUGH WALLS

Standard SAR imaging does not work

Because of refraction, multi-paths and clutter

Estimated model, work in progress

Remains:

-increase convergence area

-increase complexity of scenario

-adaptive control of sensors

DYNAMIC LOGIC / NMF

INTEGRATED INFORMATION: objects; relations;situations; behavior

Data and Signals

Dynamic Logic

combining conceptual analysis

with emotional evaluation

MODELS

- objects- relations- situations- behavioral

CLASSICAL METHODOLGYno closure

Result: Conceptual objects

signals Input: World/sceneSensors / Effectors

MODELS/templates •objects, sensors•physical models

Recognition

NMF: closurebasic two-layer hierarchy: signals and concepts

Result: Conceptual objects

signals

Input: World/scene

Sensors / Effectors

Correspondence / Similarity measures

MODELS •objects, sensors•physical models

Attention / Actionsignals Sim.signals

APPLICATION 7

• Prediction

- Financial prediction

PREDICTION

Simple: linear regression– y(x) = Ax+b– Multi-dimensional regression: y,x,b are vectors, A is a m-x– Problem: given {y,x}, estimate A,b

Solution to linear regression (well known)– Estimate means <y>, <x>, and x-y covariance matrix C– A = Cyx Cxx

-1; b = <y> - A<x>

Difficulties– Non-linear y(x), unknown shape– y(x) changes regime (from up to down)

• and this is the most important event (financial prediction)

– No sufficient data to estimate C • required ~10*dx+y

3 data points, or more

NMF/DL PREDICTION

General non-linear regression (GNLR)– y(x) = f(m|n) ym(x) = f(m|n) (Amx+bm)

– Amand bm are estimated similar to A,b in linear regression with the following change: all (…) are changed into f(m|n)(…)

For prediction, we remember that f(m|n) = f(m|x)

Interpretation– m are “regimes” or “processes”, f(m|x) determines influence of

regime m at point x (probability of process m being active)

Applications– Non-linear y(x), unknown shape– Detection of y(x) regime change (e.g. financial prediction or control)

• Minimal number of parameters: 2 linear regressions; f(m|n) are functions of the same parameters

• Efficient estimation (ML)• Potential for the fastest possible detection of a regime change

m

m

n

n

FINANCIAL PREDICTION Efficient Market Hypothesis

Efficient market hypothesis, strong: – no method for data processing or market analysis will bring

advantage over average market performance (only illegal trading on nonpublic material information will get one ahead of the market)• Reasoning: too many market participants will try the same

tricks

Efficient market hypothesis, week: – to get ahead of average market performance one has to do

something better than the rest of the world: better math. methods, or better analysis, or something else (it is possible to get ahead of the market legally)

FINANCIAL PREDICTIONBASICS OF MATH. PREDICTION

Basic idea: train from t1 to t2, predict and trade on t2+1; increment: t1->t1+1, t2->t2+1; …– Number of data points between t1 to t2 should be >> number

of parameters

Decide on frequency of trading, it should correspond to your psychological makeup and practical situation– E.g. day-trading has more potential for making (or losing) a

lot of money fast, but requires full time commitment

Get past data, split into 3 sets: (1)developing, (2)testing, (3)final test (best, in real time, paper trades)

After much effort on (1), try on (2), if work, try on (3)

DETAILS OFFINANCIAL PREDICTION

Develop “best” mathematical technique for market prediction– Takes a lot of effort– Test in “up” and “down” markets– If your simulated portfolio goes up and up smoothly (you are

constantly making money in computer simulation), look for an error• Simple errors in computing return• Include spread and commission in return• “Illegal” training: using (t2+1) information for training (or trading)

Technical details– What to optimize in training/development?

• Performance (return or cum. return), ROR = ($end - $start) / $start • Sharpe (return/risk ratio) Sh = (ROR – RORmarket) / std(RORmarket)

– Sh > 1 ok, Sh > 3 look for errors (beware)

– Include penalizing factor for free parameters (Akaike, Statistical Learning Theory (Vapnik), Ridge regression)

• Ridge regression: min [ (y(t) – p(t))2 + a (p(t) – <p(t)>)2 ]

FINANCIAL MARKET PREDICITION

Recommended Portfolios vs. Marketsportfolio gains: rec-sp = 6.2%, rec-nq = 7.4% (vs. markets sp = 1.8%, nq = -4.2% loss)

risk measures: gain/st.dev = 3.6, 4.0 (vs mkts 0.35, -.45), average exosure = 14% (vs. mkt 100%)

94.00%

96.00%

98.00%

100.00%

102.00%

104.00%

106.00%

108.00%

110.00%

12/30/2005 1/29/2006 3/1/2006 3/31/2006 5/1/2006 5/31/2006 7/1/2006

date

cu

mu

lati

ve g

ain

% (

VA

MI)

SP weeklyVAMI

NQ weeklyVAMI

RecSP+T weeklyVAMI

RecNQ+T weeklyVAMI

BIOINFORMATICS

Many potential applications– combinatorial complexity of existing algorithms

Drug design– Diagnostics: which gene / protein is responsible

• Pattern recognition• Identify a pattern of genes responsible for the condition

– Relate sequence to function• Protein folding (shape)• Relate shape to conditions

Many basic problems are solved sub-optimally (combinatorial complexity) – Alignment– Dynamic system of interacting genes / proteins

• Characterize• Relate to conditions

NMF/DL FOR COGNITIONSUMMARY

Cognition– Integrating knowledge and data / signals– Evolution from vague to crisp

Knowledge = concepts = models

Knowledge instinct = similarity(models, data)

Aesthetic emotion = change in similarity

Emotional intelligence– combination of conceptual knowledge and emotional evaluation

Applications– Recognition, tracking, fusion, prediction…

OUTLINE

• Cognition, complexity, and logic

• The Mind and Knowledge Instinct

• Language




LANGUAGE

Integration of language-data and language-models– Speech, text– Language acquisition / learning– Search engines

Language is similar to cognition – specific language data– NMF of language

LANGUAGE ACQUISITIONAND COMPLEXITY

Chomsky: linguistics should study the mind mechanisms of language (1957)

Chomsky’s language mechanisms– 1957: rule-based– 1981: model-based (rules and parameters)

Combinatorial complexity– For the same reason as all rule-based and

model-based methods

HIERARCHY OF LANGUAGE

Speech is a (loose) hierarchy of objects – Words are “made of” language sounds, phonemes– Phrases are “made of” words– …

Text is a (loose) hierarchy of objects – Letters, words, phrase,…

Meanings of language objects – Language objects acquire meanings in the hierarchy– Phonemes acquire meaning in words– Words acquire meaning in phrases– Phrases acquire meaning in paragraphs,…

APPLICATION: SEARCH ENGINE BASED ON UNDERSTANDING

Goal-instinct: –Find conceptual similarity between a query and text

Analyze query and text in terms of concepts

“Simple” non-adaptive techniques –By keywords–By key-sentences = set of words

• Define a sequence of words (“bag of words”)• Compute coincidences between the bag and the document• Instead of the document use chunks of 7 or 10 words

How to learn useful sentences?

APPLICATION: LEARN LANGUAGE UNDERSTANDING

Goal-instinct: – Find conceptual similarity between a query and text

Analyze query and text in terms of concepts

Learn and identify model-concepts in texts – Words from a dictionary– Hierarchy

• phrases made up of words, paragraphs of phrases…

Language instinct knowledge instinct

OUTLINE



• Language- NMF of language




NMF OF LANGUAGE basic two-layer hierarchy: words and phrase-concepts

Words and Concept-Models– words w(n), n = 1,…,N;

– model-phrase Mm(Sm,n), parameters Sm, m = 1, …;

– Simplistic “bag”-model: Mm(Sm,n) = Sm = {nm ; wm,1 ;wm,2 ;…wm,s}

• nm is a position of the model-center in the text {s/2… (N-s/2)}

Goal: learn phrase-models– associate sequences n with models m and find parameters Sm

– learn word-contents of phrases (and grammatical relationships)

Maximize similarity between words and models, L– Likelihood L = l(w(n))

– l(w(n)) = r(m) l(w(n) | Mm(Sm,n))

– CC: L contains MN items: all associations of words and models

n

m

DYNAMIC LOGIC (non-combinatorial solution)

Start with a large body of text and unknown phrase-models– any parameter values Sm – associate fuzzy phrase-model with its contents (words)– (1) f(m|n) = r(m) l(n|m) / r(m') l(n|m')

Improve parameter estimation– (2) Sm = Sm + f(m|n) [lnl(n|m)/Mm]*[Mm/Sm]

• ( determines speed of convergence)

– learn word-contents of phrases (and grammatical relationships)

Continue iterations (1)-(2). Theorem: NMF is converging

- similarity increases on each iteration - aesthetic emotion is positive during learning

'm

n

DIFFERENTIATION OF QUALITATIVE FUNCTIONS

Differentiation of bag-model– The “bag”-model is non-differentiable– This is a principal moment, learning non-differentiable models

requires sorting through combinations– Lead to combinatorial complexity

How to differentiate – Non-continuous, non-differentiable, qualitative functions

Also essential for the hierarchy – Higher level are made up of “bags” of lower level concepts

“QUALITATIVE” DERIVATIVE

Define fuzzy conditional partial similarity as– l(n|m) = Σs G(e(n,m,s) , m )

– e(n,m,s) is a distance between n and the word wm,s in Mm(n) that matches w(n), counted in the number of words (if no match, e(n,m,s) = S/2+1)

– Fuzziness is determined by phrase-model length, S

and matching st.dev. m = S / 3

Parameter estimation– Initialize with a large S and any values for { wm,s } – On every iteration compute e(n,m,s) and m

– From every model delete the least likely word– Reduce the phrase length S by 1– Thus, “most likely” words are gradually selected for each model– Details in Perlovsky, L.I. (2006). Symbols: Integrated Cognition and Language. Book Chapter in A. Loula, R. Gudwin, J. Queiroz, eds.,

Computational Semiotics. Idea Group, Hershey, PA.

OUTLINE



• Language




LANGUAGE vs. COGNITION

• “Nativists”, - since the 1950s- Language is a separate mind mechanism (Chomsky)- Pinker: language instinct

• “Cognitivists”, - since the 1970s- Language depends on cognition- Talmy, Elman, Tomasello…

• “Evolutionists”, - since the 1980s- Hurford, Kirby, Cangelosi…- Language transmission between generations

• Co-evolution of language and cognition

WHAT WAS FIRST COGNITION OR LANGUAGE?

How language and thoughts come together?

Conscious – “final results” ~ logical concepts

Language seems completely conscious– However, a child at 5 knows about “good” and “bad” guys– Are these conscious concepts?

Unconscious – fuzzy mechanisms of language and cognition

Logic:– Same mechanisms for L. & C.– Did not work

Sub-conceptual, sub-conscious integration

INTEGRATEDLANGUAGE AND COGNITION

Where language and cognition come together?– A fuzzy concept m has linguistic and cognitive-sensory models

• Mm = { Mmcognitive,Mm

language };

– Language and cognition are fused at fuzzy pre-conceptual level• before concepts are learned

Understanding language and sensory data– Initial models are fuzzy blobs

– Language models have empty “slots” for cognitive model (objects and situations) and v.v.

– Child’s learning

– Language participates in cognition and v.v.

L & C help learning and understanding each other – Help associating signals, words, models, and behavior

INNER LINGUISTIC FORM HUMBOLDT, the 1830s

In the 1830s Humboldt discussed two types of linguistic forms– words’ outer linguistic form (dictionary) – a formal designation– and inner linguistic form (???) – creative, full of potential

This remained a mystery for rule-based AI, structural linguistics, Chomskyan linguistics– rule-based approaches using the mathematics of logic make

no difference between formal and creative

In NMF / DL there is a difference – static form of learned (converged) concept-models– dynamic form of fuzzy concepts, with creative learning

potential, emotional content, and unconscious content

OUTLINE



• Language




HIGHER COGNITIVE FUNCTIONS

Abstract models are at higher levels of hierarchy– More vague-fuzzy, less conscious

At every level– Bottom-up signals are recognized lower-level-concepts

– Top-down signals are vague concept-models

– Behavior-actions (including learning-adaptation)

Similarity measures

Models

Action/Adaptation

Models

Action/AdaptationSimilarity measures

objects

situations

meanings

TOWARD A THEORY OF THE MIND

From Plato to physics of the mind– A mathematical theory describing “first

principles” of the mind – Corresponding to existing data and

making testable predictions

FROM PLATO TO LOCKE

Realism: – Plato: ability for thinking is based on a priori Ideas– Aristotle:

• ability for thinking is based on a priori (dynamic) Forms• an a priori form-as-potentiality (fuzzy model) meets

matter (signals) and becomes a form-as-actuality (a concept)

• actualities obey logic, potentialities do not

Nominalism– Antisthenes (contemporary of Plato)

• there are no a priori ideas, just names for similar things– Occam (14th c.)

• Ideas are linguistic phenomena devoid of reality– Locke (17th c.)

• A newborn mind is a “blank slate”

FROM KANT TO GROSSBERG

Kant: three primary inborn abilities– Reason = understanding (models of cognition)– Practical Reason = behavior (models of behavior)– Judgment = emotions (similarity)– “We only know concepts, not things-in-themselves”

Jung– Conscious concepts are developed based on inherited structures of the

mind, archetypes, inaccessible to consciousness– Realists vs. nominalists = introverts vs. extroverts

Chomsky– Inborn structures, not “general intelligence”

Grossberg– Models attaining a resonant state (winning the competition for signals)

reach consciousness

DL & Aristotle– A-priori models are vague-fuzzy and unconscious – “Understood” models in a resonant state are crisper and more

conscious

NMF AND BUDDHISM

Fundamental Buddhist notion of “Maya” – the world of phenomena, “Maya”, is meaningless deception

– penetrates into the depths of perception and cognition

– phenomena are not identical to things-in-themselves

Fundamental Buddhist notion of “Emptiness” – “consciousness of bodhisattva wonders at perception of emptiness

in any object” (Dalai Lama 1993)

– any object is first of all a phenomenon accessible to cognition

– value of any object for satisfying the “lower” bodily instincts is much less than its value for satisfying higher needs, knowledge instinct

– Bodhisattva’s consciousness is directed by the knowledge instinct

– concentration on “emptiness” does not mean emotional emptiness, but the opposite, the fullness with highest emotions related to the knowledge instinct, beauty and spiritually sublime

MIND VS. BRAIN

We start understanding how to relate the mind to the brain– Which neural circuits of the brain implement which functions

of the mind

NMF DYNAMICS

A large number of model-concepts compete for incoming signals

Uncertainty in models corresponds to uncertainty in associations f(m|n)

Eventually, one model (m') wins a competition for a subset {n'} of input signals x(n), when parameter values match object properties, and f(m'|n) values become close to 1 for n{n'} and 0 for n{n'}

Upon convergence, the entire set of input signals {n} is divided into subsets, each associated with one model-object

Fuzzy a priori concepts (unconscious) become crisp concepts (conscious) – dynamic logic, Aristotelian forms, Jungian

archetypes, Grossberg resonance Elementary thought process

CONSCIOUSNESS AND UNCONSCIOUS

Jung: conscious concepts and unconscious archetypes

Grossberg: models attaining a resonant state (winning the competition for signals) reach consciousness

NMF: fuzzy mechanisms (DL) are unconscious, crisp concept-models, adapted and matched to data are conscious

AESTHETIC EMOTIONS

Not related to bodily satisfaction

Satisfy instincts for knowledge and language – learning concepts and learning language

Not just what artists do

Guide every perception and cognition process

Perceived as feeling of harmony-disharmony– satisfaction-dissatisfaction

Maximize similarity between models and world– between our understanding of how things ought to be

and how they actually are in the surrounding world; Kant: aesthetic emotions

BEAUTY

Harmony is an elementary aesthetic emotion– higher aesthetic emotions are involved in the

development of more complex “higher” models

The highest forms of aesthetic emotion, beautiful – related to the most general and most important models– models of the meaning of our existence, of our purposiveness– beautiful object stimulates improvement of the highest models

of meaning

Beautiful “reminds” us of our purposiveness– Kant called beauty “aimless purposiveness”: not related to

bodily purposes– he was dissatisfied by not being able to give a positive

definition: knowledge instinct– absence of positive definition remained a major source of

confusion in philosophical aesthetics till this very day

Beauty is separate from sex, but sex makes use of all our abilities, including beauty

INTUITION

Complex states of perception-feeling of unconscious fuzzy processes– involves fuzzy unconscious concept-models– in process of being learned and adapted

• toward crisp and conscious models, a theory– conceptual and emotional content is undifferentiated– such models satisfy or dissatisfy the knowledge

instinct before they are accessible to consciousness, hence the complex emotional feel of an intuition

Artistic intuition – composer: sounds and their relationships to psyche– painter: colors, shapes and their relationships to psyche– writer: words and their relationships to psyche

INTUITION: Physics vs. Math.

Mathematical intuition is about– Structure and consistency within the theory– Relationships to a priori content of psyche

Physical intuition is about– The real world, first principles of its organization, and

mathematics describing it

Beauty of a physical theory discussed by physicists – Related to satisfying knowledge instinct

• the feeling of purpose in the world

WHY ADAM WAS EXPELLED FROM PARADISE?

God gave Adam the mind, but forbade to eat from the Tree of Knowledge– All great philosophers and theologists from time immemorial

pondered this – Maimonides, 12th century

• God wants people to think for themselves• Adam wanted ready-made knowledge• Thinking for oneself is difficult (this is our predicament)

Today we can approach this scientifically – Rarely we use the KI– Often we use ready-made heuristics, rules-of-thumb– Both are evolutionary adaptations– Cognitive effort minimization is opposite to the KI

2003 Nobel Prize in Economics – People’s choices are often irrational

GOD, SNAKE, and fMRI

Majority often make irrational, heuristic choices (CEM-type)

Stable minority is rational (KI-type)

fMRI – KI-type think with cortex (uniquely human)– CEM-type think with amygdala (animals)

God demands us being humans

“Snake’s apple” pulled Adam back to animals

SYMBOL

“A most misused word in our culture” (T. Deacon)

Cultural and religious symbols– Provoke wars and make piece

Traffic Signs

SIGNS AND SYMBOLSmathematical semiotics

Signs: stand for something else– non-adaptive entities (mathematics, AI)– brain signals insensitive to context (Pribram)

Symbols– general culture: deeply affect psyche– psychological processes connecting conscious

and unconscious (Jung)– brain signals sensitive to context (Pribram)– signs (mathematics, AI): mixed up– processes of sign interpretation

NMF: mathematics of symbol-processes– elementary thought process

NMF THEORY OF SYMBOLS -mathematical semiotics-

Semiotics studies symbol-content of culture

Example: a written word "chair" – The mind interprets it to refer to something else: an entity in the world, a

specific chair, or the concept "chair" in the mind– The mind, or an intelligent system is an interpreter, the written word is a

sign, the real-world chair is a designatum, and the concept in the interpreter's mind, the internal representation of the results of interpretation is called an interpretant of the sign

– The essence of a sign is that it can be interpreted by an interpreter to refer to something else, a designatum

This is a simplified description of a thinking process, called semiosis– its mechanism is given by the elementary thought process

Elementary thought process involving consciousness and unconscious, concepts and emotions, is a dynamic symbol process – a much more complicated entity than was originally envisioned by founders

of “symbolic AI”

SYMBOLS and “SYMBOLIC AI”

Founders of “symbolic AI” believed that by using “symbolic” mathematical notations they would penetrate into the mystery of mind– But mathematical symbols are just notations (signs)– Not psychic processes

This explains why “symbolic AI” was not successful

This also illustrates the power of language over thinking– Wittgenstein called it

• “bewitchment (of thinking) by language”

SYMBOLIC ABILITY

Integrated hierarchies of Cognition and Language– High level cognition is only possible due to language

– Language is only possible due to cognition

Similarity Action

ActionSimilarity

Similarity Action

ActionSimilarity

cognition language

M

M M

M

grounded in real-world objects

grounded in language

grounded in language

OUTLINE



• Language



• Future directions- Evolution of languages and cultures

CULTURE AND LANGUAGE

• Animal consciousness–Undifferentiated, few vague concepts

–No mental “space” between thought, emotion, and action

• Evolution of human consciousness and culture –More differentiated concepts

–More mental “space” between thoughts, emotions, and actions

–Created by evolution of language

• Language, concepts, emotions–Language creates concepts

–Still, colored by emotions

16-Sep-05 121

EVOLUTION OF CULTURES

• The knowledge instinct–Two mechanisms: differentiation and synthesis

• Differentiation –At every level of the hierarchy: more detailed concepts–Separates concepts from emotions

• Synthesis –Knowledge has to make meaning, otherwise it is useless–Diverse knowledge is unified at the higher level in the hierarchy–Connects concepts and emotions

Connect language and cognitionConnect high and low: concepts acquire meaning at the next level

16-Sep-05 122

DYNAMICS OF DIFFERENTIATION AND

SYNTHESIS• Differentiation, D

–New knowledge comes from differentiating old knowledge,Speed of change of D ~ D

–Differentiation continues if knowledge is useful (emotional)Speed of change of D ~ - S

–Differentiation stops if knowledge is “too” emotionalSpeed of change of D ~ 0, if S id “too large”

• Synthesis, S–Emotional value of knowledge–Emotions per concept diminish with more concepts

Speed of change of S ~ -D–Synthesis grows in the hierarchy (H)

Speed of change of S ~ H

16-Sep-05 123

CULTURAL STATES CAN BE MEASURED

Differentiation– Number of words

Synthesis– Emotions per word

Hierarchy – Social, political, cultural, language

“Material” measures – Demographics, geopolitics, natural resources…– Ignore for a moment

MODELING “spiritual aspects” ofCULTURAL EVOLUTION

Differentiation, synthesis, hierarchy

dD/dt = a D G(S); G(S) = (S - S0) exp(-(S-S0) / S1)

dS/dt = -bD + dH

H = H0 + e*t

DYNAMIC CULTURE

Average synthesis, high differentiation; oscillating solutionKnowledge accumulates; no stability

TRADITIONAL CULTURE

High synthesis, low differentiation; stable solutionStagnation, stability increases

TERRORIST’S CONSCIOUSNESS

Ancient consciousness was “fused”– Concepts, emotions, and actions were one

• Undifferentiated, fuzzy psychic structures

– Psychic conflicts were unconscious and projected outside

• Gods, other tribes, other people

Complexity of today’s world is “too much” for many– Evolution of culture and differentiation

• Internalization of conflicts: too difficult

– Reaction: relapse into fused consciousness• Undifferentiated, fuzzy, but simple and synthetic

The recent terrorist’s consciousness is “fused”– European terrorists in the 19th century– Fascists and communists in the 20th century– Current Moslem terrorists

INTERACTING CULTURES

Knowledge accumulation + stability

1) Early: Dynamic culture affects traditional culture, no reciprocity2) Later: 2 dynamic cultures stabilize each other

FUTURE SIMULATIONS OF EVOLUTION

Genetic evolution simulations (1980s - )– Used basic genetic mechanisms – Artificial Life, evolution models: Bak and Sneppen,

Tierra, Avida

Evolution of cultural concepts – Genes vs. memes (cultural concepts)– Evolution of concepts vs. evolution of genes

• Culture evolves much faster than genetic evolution • Human culture is <10,000 years, likely, no genetic

evolution (?)– Evolution of languages– Concepts evolve from fuzzy to crisp and specific– Concepts evolve into a hierarchy– Concepts are propagated through language

MECHANISMS OF CONCEPT EVOLUTION

Differentiation, synthesis, and language transmission

Differentiation– Fuzzy contents become detail and clear– A priori models, archetypes are closely connected to unconscious

needs, to emotions, to behavior• Concepts have meanings

Cultural and generational propagation of concepts through language– Integration of language and cognition is not perfect

• Language instinct is separate from knowledge instinct

Propagation of concepts through language– A newborn child encounters highly-developed language– Synthesis: cognitive and language models {MC, ML} are connected

individually– No guarantee that language model-concepts are properly integrated

with the adequate cognitive model-concepts in every individual• And we know this imperfection occurs in real life• Meanings might be lost• Some people speak well, but do not quite understand and v.v.

SPLIT BETWEEN CONCEPTUAL AND EMOTIONAL

Dissociation between language and cognition – Might prevail for the entire culture

Words maintain their “formal” meanings– Relationships to other words

Words loose their “real” meanings– Connection to cognition, to unconscious and emotions

Conceptual and emotional dissociate– Concepts are sophisticated but “un-emotional”– Language is easy to use to say “smart” things

• but they are meaningless, unrelated to instinctual life

CREATIVITY

At the border of conscious and unconscious

Archetypes should be connected to consciousness– To be useful for cognition

Collective concepts–language should be connected to – The wealth of conceptual knowledge (other concepts)– Unconscious and emotions

Creativity in everyday life and in high art– Connects conscious and unconscious

Conscious-Unconcious ≠ Emotional-Conceptual– Different slicing of the psyche

DISINTEGRATION OF CULTURES

Split between conceptual and emotional – When important concepts are severed from emotions – There is nothing to sacrifice one’s life for

Split may dominate the entire culture– Occurs periodically throughout history – Was a mechanism of decay of old civilizations– Old cultures grew sophisticated and refined but got

severed from instinctual sources of life• Ancient Acadians, Babylonians, Egyptians, Greeks,

Romans…

– New cultures (“barbarians”) were not refined, but vigorous• Their simple concepts were strongly linked to instincts,

“fused”

EMOTIONS IN LANGUAGE

• Animal vocal tract–controlled by old (limbic) emotional system–involuntary

• Human vocal tract–controlled by two emotional centers: limbic and cortex–Involuntary and voluntary

• Human voice determines emotional content of cultures–Emotionality of language is in its sound: melody of speech

16-Sep-05 135

LANGUAGE:EMOTIONS AND CONCEPTS

• Conceptual content of culture: words, phrases–Easily borrowed among cultures

• Emotional content of culture–In voice sound (melody of speech)–Determined by grammar–Cannot be borrowed among cultures

• English language (Diff. > Synthesis)–Weak connection between conceptual and emotional (since 15 c)–Pragmatic, high culture, but may lead to identity crisis

• Arabic language (Synthesis > Diff.)–Strong connection between conceptual and emotional–Cultural immobility, but strong feel of identity (synthesis)

16-Sep-05 136

SYNTHESIS

People cannot live without synthesis– Feel of wholeness– Meaning and purpose of life

Creativity, life, and vigor requires synthesis– Emotional and conceptual, conscious and unconscious– In every individual

• Lost synthesis and meaning leads to drugs and personal disintegration– In the entire culture

• Lost synthesis and meaning leads to cultural disintegration

Historical evolution of consciousness– From primitive, fuzzy, and fused to differentiated and refined– Interrupted when synthesis is lost– Differentiation and synthesis are in opposition, still both are required– Example: religion vs. science

• Religious synthesis empowered human mind (15 c) and created conditions for development of science (17 c)

• Scientific differentiation destroyed religious synthesis• Evolution of our culture requires overcoming this split, and it is up to us, scientists and engineers

Individual consciousness – Combining differentiation and synthesis– Jung called individuation, “the highest purpose in every life”

MECHANISM OF SYNTHESIS

Integrating the entire wealth of knowledge– Undifferentiated knowledge instinct “likelihood maximization”

• Global similarity– Differentiated knowledge instinct

• Highly-valued concepts• Local similarity among concepts

Highly valued concepts acquire properties of instincts– Affect adaptation, differentiation, and cognition of other concepts– Generate emotions, which relate concepts to each other

Differentiated knowledge instinct – An emergent hierarchy of concept-values – Differentiated emotions connect diverse concepts– We need huge diversity of emotions to integrate conceptual

knowledge => synthesis

DIFFERENTIATION OF EMOTIONS

Historical evolution of human consciousness– Animal calls are undifferentiated

• concept-emotion-communication-action– Ancient languages are highly emotional (Humboldt, Levy-

Brule)

Language evolved toward unemotional differentiation– Nevertheless, most conversations have little conceptual

content• From villages to corporate board-rooms, people talk to

establish emotional contact• Human speech affects recent and ancient emotional centers

– Inflections and prosody of human voice appeals directly to ancient undifferentiated emotional mechanisms

– Accelerates differentiation, but endangers synthesis

Music evolved toward differentiation of emotions– At once: creates tensions and wholeness in human soul

ROLE OF MUSIC IN EVOLUTION OF THE MIND

Melody of human voice contains vital information– About people’s world views and mutual compatibility – Exploits mechanical properties of human inner ear

• Consonances and dissonances

Tonal system evolved (14th to 19th c.) for – Differentiation of emotions– Synthesis of conceptual and emotional– Bach integrates personal concerns with “the highest”

Pop-song is a mechanism of synthesis– Integrates conceptual (lyric) and emotional (melody)– Also, differentiates emotions– Bach concerns are too complex for many everyday needs– Human consciousness requires synthesis immediately

Rap is a simplified, but powerful mechanism of synthesis– Exactly like ancient Greek dithyrambs of Dionysian cult

EVOLUTION vs. INTELLIGENT DESIGN

• Science causal mechanisms

• Religion teleology (purpose)

• Wrong! –In basic physics causality and teleology are equivalent–The principle of minimal energy is teleological–More general, min. action (min. Lagrangian)

• The knowledge instinct –Teleological principle in evolution of the mind and culture –Dynamic logic is a causal law equivalent to the KI–Causality and teleology are equivalent

16-Sep-05 141

PREDICTIONS AND TESTING of NMF/DL theory of the mind

Methods of experimental testing– Neural, psychological, and psycholinguistic labs– Emerging method: simulation of multi-agent evolving systems

General neural mechanisms of the thought process– includes neural mechanisms for bottom-up (sensory) signals, top-down

(“imagination”) model-signals, and the resonant matching between the two – confirmed by neural and psychological experiments

Adaptive modeling abilities – well studied: adaptive parameters are synaptic connections

Instinctual learning mechanisms

Ongoing and future research will confirm, disprove, or suggest modifications to

– similarity measure as a foundation of knowledge and language instincts– mechanisms of model parameterization and parameter adaptation– dynamics of fuzziness during perception/cognition/learning– mechanisms of language and cognition integration– mechanisms of differentiation and synthesis

CURRENT AND FUTURE RESEARCH DIRECTIONS

Physics of the mind– Higher cognitive functions– Intuition, beautiful, sublime, conscious, unconscious,

creativity– Cognition and language

Inverse problems / control for complex physical systems

Robotic systems– Integrated autonomous control, sensing, communication

cognition & language– Collaborative: robots learn from humans– Evolving (improving not obsolete)

Semantic web Interactive environment Evolution of languages and cultures

– Role of music in cognition and evolution– Better understanding among peoples

THE END

Can we describe mathematically and build a simulation model for evolution of all of these abilities?

Can we build robotic systems understanding us, collaborating with us?

PUBLICATIONS

280 publications

OXFORD UNIVERSITY PRESS(2001; 3rd printing)

2007

Neurodynamics of High Cognitive Functionswith Prof. Kozma, Springer

Sapient Systemswith Prof. Mayorga, Springer

Future:

The Knowledge InstinctBasic Books

BACK UP

Why the mind and emotions?

NMF vs. inverse problems

NMF vs. biology of eye

Cognition and understanding

Intelligent agents

WHY MIND AND EMOTIONS?

A lot about the mind can be explained: concepts, instincts, emotions, conscious and unconscious, intuition, aesthetic ability,…– but, isn’t it sufficient to solve mathematical equations or to code a

computer and execute the code?

A simple yet profound question– the answer is in history and practice of science– Newton laws do not contain all of the classical mechanics– Maxwell equations do not exhaust radars and radio-communication– a physical intuition about the system is needed – an intuition about NMF was derived from biological, linguistic,

cognitive, neuro-physiological, and psychological insights into human mind

Practical engineering applications of DL require biological, linguistic, cognitive, neuro-physiological, and psychological insights in addition to mathematics

NMF vs. BIOLOGY OF EYE

Human eye is a part of the brain - Integrated sensor-processor system in both design and in operations

• multi-layer hierarchical system• integrated adaptive optimized resource allocation

- Feedback from higher layers to lower layers is essential • eye-brain neural pathway contains more feedback connections than feedforward

ones- Perception involves matching of retinal projections and model projection-primings

Adaptive, joint optimization of sensor-processor system network - Based on a hierarchy with feedback among layers and modules

NMF hierarchy with feedback - Every layer has 5 basic modules/elements: (1) incoming signals (structured at lower layer, unstructured at the current layer) (2) models(3) similarity measure between signals and models (knowledge instinct)(4) adaptation mechanism(5) outgoing signals (a structure: sign-concept)

NMF VS. INVERSE SCATTERING

Inverse Scattering in Physics– reconstruction of target properties by “propagating back” scattered fields

– usually complicated, ill-posed problems (exception: CATSCAN)

Biological systems (mind) solve this problem all the time– by utilizing prior information (in feedback neural pathways)

Classical Tikhonov’s inversion cannot use knowledge– regularization parameter () is a constant – Morozov’s modification can utilize prior estimate of errors for

Inverse Scattering using NMF– can utilize any prior knowledge ( became an operator)– utilizes prior knowledge adaptively ( depends on parameters)

COGNITION AND UNDERSTANDING

Incoming signals, {x,w} are associated with model-concepts (m)– creating phenomena (of the NMF-mind), which are understood

as objects, situations, phrases,…– in other words signal subsets acquire meaning (e.g., a subset

of retinal signals acquires a meaning of a chair)

Several aspects of understanding and meaning– concept-models are connected (by emotional signals) to

instincts and to behavioral models that can make use of them for satisfaction of bodily instincts

– an object is understood in the context of a more general situation in the next layer consisting of more general concept-models (satisfaction of knowledge instinct)

• each recognized concept-model (phenomenon) sends (in neural terminology: activates) an output signal

• a set of these signals comprises input signals for the next layer models, which ‘cognize’ more general concept-models

• this process continues up and up the hierarchy towards the most general models: models of universe (scientific theories), models of self (psychological concepts), models of meaning of existence (philosophical concepts), models of a priori transcendent intelligent subject (theological concepts)

NMF AND INTELLIGENT AGENTS

Agents are– Autonomy and interaction, communication – Goal-oriented, functional– Sense environment, extracts information – use knowledge, produce new knowledge– embody the concept of life and intelligence

Every NMF model-concept is an ‘agent’– Goal: improve its fitness to the data (improve knowledge)– Significantly autonomous, communicates with other agents– Receives signals from sensors or other agents – Competitive-cooperative interaction (learning)

• cooperation within a model, competition among models

Higher level: Culture, society of NMF agents (minds)– Learn from each other– Communicate, evolve language, culture– Applications: collaborative systems

leonid perlovsky visiting scholar, harvard university technical advisor, afrl

Documents

nmf of language3

clutter complexity nmf

nmf dynamics

modeling field theory

clutter models2

evolution of concepts

integration of cognition

fusion example