polynomial discrete time cellular neural networks eduardo gomez-ramirez † giovanni egidio...
TRANSCRIPT
Polynomial Discrete Time Cellular Neural Networks
Eduardo Gomez-Ramirez † Giovanni Egidio Pazienza‡
† LIDETEA, POSGRADO E INVESTIGACION Universidad La Salle – México, D.F. ‡ Department d’Electronica, EALS
Universitat “Ramon Llull” – Barcelona, Spain
Outline Cellular Neural Networks (CNN)
Introduction and Objective Genetic Algorithms (GA) Polynomial Discrete Time CNNs
(PDTCNNs) XOR Problem Game of Life
Learning vs Design Conclusions and future workIntro CNN & GA Polyn. CNN XOR GoL Conclusions
CNN: Introduction CNN for complex task (linearly
nonseparable data)
Multilayer CNNs Include more degrees of freedom for the
output state of each layer Search in a finite set of templates
Single layer: Polynomial CNNs
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Improve the representation power of a single layer CNN including a simple nonlinear term to solve problems with linearly nonseparable data (XOR)
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Objective
CNN: mathematical model
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
The simplified mathematical model is:
where xc is the state of the cell, uc the input and yc the output
iubtyatxdt
dx dcd
dcd
cc
)()(
)1)(1)((2
1 txtxy ccc
CNN: Activation Function
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
CNN: Block Diagram
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
CNN: Discrete Model
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Computing x(∞), the model can be represented as
iubnyanx dcd
dcd
c )()(
0)(,1
0)(,1)(
nx
nxny
c
c
c
using the following activation function
Steps: Crossover C(Fg) Mutation M(*) Adding random parent Ag()
GA: main steps proposed
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
dc
bc
da
ba
FC
dcMbaMM
MF
g
g 212
1 ,,
GA: Crossover
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
125.0125.
000
000
A
125.125.0
000
000
B I=0
125.00
000
000
A
125.125.125.
000
000
B I=0
2121
2211
2222
2121
2211
1111
1
cccc
bbbb
aaaa
cccc
bbbb
aaaa
P
Individual 1
Individual 2
GA: Crossover
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
a1
b1
c1
a2
b2
c2
GA: Mutation
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
mij
mijmij PrF
PrFPFM
)(
)(),(
where rU(0,1) is a random variable with uniform distribution defined on a probability space (,,P),
GA: Mutation (resolution)
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
'2
'1
'2
'1
'2
'2
'1
'1
'2
'2
'2
'2
'2
'1
'2
'1
'2
'2
'1
'1
'1
'1
'1
'1
'1
cccc
bbbb
aaaa
cccc
bbbb
aaaa
P
125.0125.
000
000
A
125.125.0
000
000
B I=0
125.00
000
000
A
125.125.125.
000
000
B I=0
Individual 1
Individual 2
a1
b1
c1
a2
b2
c2
Population=sons + sons mutated
g
g
g
C FA
M C F
ggn
pgg AOnAS pmin,
GA: Selecting Parents
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
)(1 wF
FF
g
gg
GA: Adding Random Parent
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
THEOREM 1: (Weierstrass’s Approximation Theorem)
Let g be a continuous real valued function defined on a closed interval [a,b]. Then, given any positive, there exists a polynomial y (which may depend on ) with real coefficients such that:
For every x [a,b].
Polynomial Discrete Time Cellular Neural Network
)()( xyxg
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
THEOREM 2 *: Any Boolean Function of n-variables can be realized using a Polynomial Threshold gates of order sn.The quadratic threshold gate can be defined:
And s is the number of inputs and T is the threshold constant.
Polynomial Discrete Time Cellular Neural Network
otherwise
Txxwxwify
n
i
n
ijiijii
0
11 1
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
* N. J. Nilsson. The Mathematical Foundations of Learning Machines. McGraw Hill, New York, 1990.
PDTCNN: the model (I)
cdd
cNd
dcd
cNd
dcd
c iyuguBkyAkxrr
),()()()()(
0)1(1
0)1(1))1(()(
kxf
kxifkxfky
c
ccc
)0(
)0(
)0(
),(
)(
)(
)(
cd
cNd
cd
cd
cNd
cd
dd
cNd
cd
dd
yuP
uyP
yuP
yug
r
r
r
)(
)(
)(
),(
)(
)(
)(
kyuP
ukyP
kyuP
yug
cd
cNd
cd
cd
cNd
cd
dd
cNd
cd
dd
r
r
r
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
PDTCNN: the model (II)
)()(),()()(
kuQkyPyug d
cNd
cd
d
cNd
cd
dd
rr
cdd
cNd
dcd
cNd
dcd
c iyuguBkyAkxrr
),()()()()(
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
PDTCNN: Solving XOR problemSome papers: Z. Yang, Y. Nishio, A. Ushida,
Templates and algorithms for two-layer cellular neural networks. IJCNN’02, 2002.
F. Chen, G. He, G. Chen & X. Xu,
Implementation of Arbitrary Boolean Functions via CNN. CNNA’06, 2006.
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
PDTCNN:Solving XOR problem M. Balsi, Generalized CNN: Potentials of
a CNN with Non-Uniform Weights. CNNA-92, 2002 .
E. Bilgili, I. C. Göknar and O. N. Ucan, Cellular neural network with trapezoidal activation function. Int. J. Circ. Theor. Appl., 2005
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Learning parameters Initialpop=20000 Number of fathers=7 Maximum number of random
parents to be add = 3 Kpro=0.8 Increment=1 Mutation Probability=0.15
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
PDTCNN:First Scheme U:uij=xijxij+1
1401
150140
iP
BA
0300
010030
iP
BA
0100
010010
iP
BA
)()()( 1 kyuukykx ccccc
)(
)(
)(
),(
)(
)(
)(
kyuP
ukyP
kyuP
yug
cd
cNd
cd
cd
cNd
cd
dd
cNd
cd
dd
r
r
r
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
PDTCNN:Second Scheme U:uij=xijyij
2281
000000
iP
BA
b)
1151
000001
iP
BAc)
0 1 0 0 0 0
0 1 0 0
A B
P i
)()()( kyukykx cccc
)0(
)0(
)0(
),(
)(
)(
)(
cd
cNd
cd
cd
cNd
cd
dd
cNd
cd
dd
yuP
uyP
yuP
yug
r
r
r
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
The Game of Life (I) The Game of Life (GoL) is a totalistic cellular
automaton consisting in a two-dimensional grid cells, that may be either alive (black) or dead (white).
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
The Game of Life (II) The state of each cell varies according to the
following rules: Birth: a cell that is dead at time t becomes
alive at time t + 1 only if exactly 3 of its neighbors were alive at time t;
Survival: a cell that was living at time t will remain alive at t + 1 if and only if it had exactly 2 or 3 alive neighbors at time t.
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
The Game of Life (III) Every sufficient well-stated mathematical problem can be reduced to a question about Life;
It is possible to make a life computer (logic gates, storage etc.);
Life is universal: it can be programmed to perform any desired calculation;
Given a large enough Life space and enough time, self-reproducing animals will emerge...
The whole universe is a CA! (E.Fredkin, MIT).Intro CNN & GA Polyn. CNN XOR GoL Conclusions
The Game of Life – NOT gate
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
A
CNN & GoL Multilayer CNN (Chua, Roska) – 1990 Activation function (Chua, Roska) –
1990 CNN-UM (Roska,Chua) -1990 CNN Universal Cells (Dogaru, Chua) –
1999
Simplicity vs. Computational power
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Polynomial CNN (I)
),()()( ddede
ede
d yugiubnyanx
0)(,1
0)(,1)(
nx
nxny
d
d
d
What’s g(ud,yd)?
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Polynomial CNN (II) In the simplest case g(ud, yd) is a second
degree polynomial, whose general form is
2
0
2 ))()()()((),(i
iedei
iedei
dd yqupyug
200 )()()1()( ed
eed
e yqp
)()()()( 11ed
eed
e yqup
)1()()()( 22
2ed
eed
e qup
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Polynomial CNN (III)
Thanks to some considerations we find that
)()()()()()()( 12
0ded
eed
eed
eed
ed yupyqiubnyanx
000
00
000
caA
ppp
pcp
ppp
bbb
bbb
bbb
B
000
00
000
0 0cqQ
ppp
pcp
ppp
ppp
ppp
ppp
P
111
111
111
1
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Polynomial CNN (IV)
pcppccccd uypbuypbnx )()()(
iyqya cccc 2
uc and appear in the state equation
direct link with totalistic Cellular Automata
pu
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
GoL: Rules (I)
GoL: Rules (II)
Rule 1: a cell will be alive at least 3 of the 9 cells in its 3 × 3 neigh. are alive
Black pixel= +1
White pixel= -1pixel centr. = 1 (black)
Σ neigh. = -2 (5 w, 2 b)
next state = -1 (white)
GoL: Rules (III)
Rule 2: a cell will be alive if at most 3 of its 8 neighbors are alive
Black pixel= +1
White pixel= -1pixel centr. = 1 (black)
Σ neigh. = -2 (5 w, 2 b)
next state = 1 (black)
Design algorithm (I) First iteration: we try to perform the first rule (a
cell will be alive at least 3 of the 9 cells in its 3 × 3 neigh. are alive)
If Y(0)=0
bc=1 bp=1 i=3
pcppccccd uypbuypbnx )()()(
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Design algorithm (II) Second iteration: we try to accomplish the
second rule (a cell will be alive if at most 3 of its 8 neighbors are alive)
pcpcccd uypuypnx )1()1()(
32 cccc yqya
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Design algorithm (III) Hyp: pc=0
Templates found using learning
Coming soon...
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Conclusions (I) In general:
In some cases it is possible to reduce a multilayer DTCNN to a single layer PDTCNN
Thanks to the GoL we can explore the capacity of PDTCNNs for Universal Machine
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Conclusions (II) About learning:
The resolution used reduces the search space
The step “Add random parent” improves the behavior to avoid local minimas
About design We give a simple algorithm to design
templates for the Polynomial CNN
Future Work Implementations of mathematical
morphology functions with PDTCNNs
Intro CNN & GA Polyn. CNN XOR GoL Conclusions
Polynomial Discrete Time Cellular Neural Networks
Eduardo Gomez-Ramirez Giovanni Egidio Pazienza
[email protected]@salle.url.edu