Математические методы распознавания образов (ММРО-13):...
TRANSCRIPT
-
...
-
FORECSYS
-13
, . ,30 6 2007
13- ,
15-
, 2007
-
004.85+004.89+004.93+519.2+519.25+519.7 ??????????????
??
?? .
13- : . .: , 2007. ??? .
ISBN ???????????
13- - , - 15- , ... ( 07-07-06050) - Forecsys.
, 1983 ., , , , , .
004.85+004.89+004.93+519.2+519.25+519.7 ??????????????
ISBN ??????????? c , 2007c , 2007c : ., 2007
-
: ,
. : , .-. : , ..-..
: , ..-.. , .... , ... , .... , ..-.. , ..-.. , ..-.. , ..-..
: , .-.
. : , ..-.. : , ..-..
: , , .-. , .-. , ... , .... , .... , .... , ...
:
:
-
. . . 5
. . . . . . . . 77
. . . . . . . 247
. . . . . . . . . . . . . . 275
. . . . . . 451
. . . . . . 569
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645
. . . . . . . . . . . . . . . . . . . . . 663
-
: TF (Theory and Fundamentals)
.
- .
.
.
.
.
.
.
.
-
6 (TF)
-
(TF) 7
.., ..
[email protected], [email protected]
,
- , [1, 2]. , [2], . - , (). , -.
-
f (x) =12
k
i=1
ifi(x), fi(x) = exp
( 122
(x i)2),
x R, k = 2, 2 , i i- , i , i (0, 1),k
i=1 i = 1. ,
() [3]. f(x) -
f x(x) = 0, fxx(x) = 0. f(x)
, - , . . [3]. f(x) [4]. 1 < 2.
1. k = 2 6 2 f(x) , , = (2 1)1. 2. k = 2 > 2, 1 = 2 f(x) xc =
12 (1 + 2) .
1 2 : k = 2 = 2, 1 = 2 xc1 == 12 (1+2) f(x), - 1 = 2, 2 = 1+2, xc1 = 1+ f x(x) f
xx(x).
xc2 k = 3, 1 = 2.446, 2 = 0, 3 = 2.446,1 = 0.4, 2 = 0.2, xc2 = 0.
-
8 (TF) .., ..
3. k = 2 > 2, 1 6= 2 f(x) , ln(112 )
> 12+ 2 ln(12
(+
2 4
)). (1)
, , - (1), 1, 2 m f(x) 1 6 m 6 3. m = 2, f(x), , .
k = 2 , 1, 2 - 15 , 1() == | ln(12)1|; 2(), (1), = 2 1. . 1 1, 2, . 15 1 2, 1, 2.
, = a2 + b + c - a, b, c ,
= 0.3272 + 3.867 3.738, (2)
= 0.101, - = 0.024, - 1 = 0.124.
1, 2, (1), (2) 1, 1 = 2 ,
ln(1
12
) = 0.8272 + 2 ln[21(+
2 4)] 3.867+ 3.738, (3)
(3) - f(x).
(), -
P{| | 6 t1
}> 1 t2,
t = 3, 1 = 0.124
P{ 0.372 < < + 0.372
}> 0.888. (4)
1 = 2 , 1 (3),(4)
P{d+ 3.366 < 1 < d+ 4.110
}> 0.888. (5)
-
(TF) 9
. 1. 1(), 2() ().
(5) 0.888 - f(x), f(x) , .
, > 2 1 6= 2 0.888 f(x) ,
ln(112 ) > d+ 4.110,
,
ln(112 ) < d+ 3.366.
, - .
[1] ., . . .: , 1975.
[2] Carreira-Perpinan M.A., Williams C. On the Number of Modes of a GaussianMixture. Inform. Res. Report EDI-INF-RR-0159. School of Inf. Univ. ofEdinburg, 2003.
[3] .., .., - .. - . .: , 1982. 304 .
[4] .., .. // , 2004. . 44, 5. . 838846.
-
10 (TF) .., ..
.., ..
[email protected], [email protected]
, . ..,
- , - [1, . 8792], - , , - , , - , .
, , , , -, , , , . - ., xyz, xyz, xy , z , , -: xy(z z) xy, z . , : - , , . ( ) - ,, . () x y xyxyxy, x y : xy xy xy xy. - () .
, - , - . x y - Axy ( x y) :
Axy Vxy Vxy Vxy, Vxy xy-, Vxy xy-,Vxy xy-.
-
(TF) 11
xy- - .
Axy VxVxy Vy, - [1, . 91], - VxVx Vy Vy, . . , x-, x-, y-, y- , Vxy VxVxy Vy x y, x y. , - Vxy, [3], - .
Exy x y Axy y: Exy Axy Vxy VxVy. Ixy x y, xy, -, : Ixy inv(Vxy) Vxy,, VxVx Vy Vy, Ixy Vxy Vx Vy. -, Oxy - Axy Oxy Vxy Vx Vy Ixy. , - A I. -: Axy, Axy, Ixy, Ixy.
- - [4] (). .
- ( ). , Barbara:
Axy Ayz Vxy Vyz V(xyyz) V(xyzxyzxyzxyz) V(xy xz yz) Vxy Vxz Vyz Vxz Axz.
, , - , ., Axy Ayz . Axy - Ixy Vxy Ixy Vxy.
Ixy Ayz Vxy Vyz Vxy(Vxyz Vxyz
) .
Ixy Ayz Vxy Vyz Vxy(Vxyz Vxyz
)
Vxy Vxyz Vxyz Vxz Ixz.
-
12 (TF) .., ..
@
@@@
@@@@
@@@@
@@@@
++
+
++0 0 0
0 0
. 1. .
, () , , , . , , .
[4], . - () : + -, , , . x, y, z, . , Vxyz (+ + +), Vxy (+ + +0),Vxz (+0+), Vxy (+0), Vxyz (++), Vxz (+0).
, , - ( , . 1) . ,
Axy Ayz Vxy Vyz (++0) (0+) (+ 0+) Vxz Axz.
, - . ,
Ixy Ayz (+0) inv(0+) (+0) (+0 +) (+ 0) Ixz.
( ) , - .
- , . , 19, 15, 128.
-
(TF) 13
[1] .. . .: . 1998.
[2] .. //- . . . 8 (43). .: -, 2003. . 317327.
[3] .. . .: - - -, 1976. . 8.
[4] .., .. - // -12. .: -, 2005. . 4042.
..
,
- , - .
- .
, - , , , , , . . , , .
, , . . -, , .
, ( - , ).
: - , - .
, , , ,
-
14 (TF) ..
, , ., - . , - , .
: - GPS, , -, , , .
, , , - - . NP- . , , - , -, , () .
, - , , , , - .
- ( ) , , - - (), ( , ). - ( ), -, . . .
, , - , backpropagation, -.
, - - . , ( -), , , , . - , - , - .
-
(TF) 15
( ), , -, .
, , - NP- : ; , .
-. [14], .
- . (, , -) , - . - .
.
1. -, , --, ,, .
2. , - .
3. ,, , .
4. , -, .
, . , .
[1] .., .. - // 2-
-
16 (TF) .., .., ..
-. , 2000. . 166170.
[2] .., .. - IEEE AIS03, CAD-2003 ( ) .1, .: ,2003. C. 208213.
[3] .., .. - // -, -11, , 2003. . 261263.
[4] .., .. // - . .: , 2006. . 280286.
.., .., ..
[email protected], [email protected], [email protected]
, ,
- . .
y Y = {1, 1}, x X = Rn. - () S = {z1 = (x1, y1), . . . , zm = (xm, ym)},(z1, . . . , zm) Zm = (X Y )m. Si = S \ {zi},i = 1, . . . ,m, S . - X R, . , u - g, g = [u]. , , sign(g), g .
, - S S . - , - ( ) S Si, i = 1, . . . ,m. S = sign(f) = sign([w]) iS = sign(f
i) = sign([wi]) - S Si .
-
(TF) 17
c : R2 R
c(y, y) =
1, yy 6 0;1 yy, 0 6 yy 6 1;0, yy > 1.
(1)
S
Rem(f) =1
m
m
i=1
c(f(xi), yi
),
Rlo(f) =1
m
m
i=1
c(f i(xi), yi
).
1. - - , S Zm, (x, y) S i = 1, . . . ,m
[w](x) [wi](x) 6 .
RVM [1] -
y = sign(g(x)) = sign([u](x)) = sign
( m
i=1
uiK(x,xi)
),
u Rm, K(, ) , . f = [w] ( f i = [wi]),
S ( Si), - g = [u], ,
1
m
m
k=1
log(1 + eykg(xk)
)+ uu,
1
m
k 6=ilog
(1 + eykg(xk)
)+ uu,
= diag(1, . . . , m), i > 0, i = 1, . . . ,m - .
2 ( [2]). F : Rm R . , F (g) F g, F (g) > F (g) + g g,F (g). g g dF (g
,g) , F (g) F (g) g g,F (g) > 0. 1 ( [2]). N
([u]
)= uTu.
dN (f, fi) + dN (f
i, f) 61
m
f(xi) f i(xi).
-
18 (TF) .., ..
1 RVM.
2. RVM :
dN (f, fi) + dN (f
i, f) = 2 12 (w wi)
2.
, ,
12 (w wi)2 6 1
2m
f(xi) f i(xi).
mm-(K(xi,xj)
)mi,j=1
K.
3. - RVM:
f(xj) f i(xj) 6
K 12
12 (w wi).
4. - -
= 12mK 12
2. Rlo(f)Rem(f)
6 .
, 07-01-00211, 05-01-00332.
[1] Tipping M.E. Sparse Bayesian Learning and the Relevance Vector Machines //Journal of Machine Learning Research. 2001. Vol. 1, 5. P. 211244.
[2] Bousquet O., Elisseeff A. Stability and Generalization // Journal of MachineLearning Research. 2002. Vol. 2, 3. P. 499526.
C
.., ..
, ,
- , . - n- . - , (
-
C (TF) 19
) . , - . , ( - ) ,, , ( ). (. [1]) - .
1. Sn(), - , .
2. Sn() - , , . .Sn() =
S
n().
3. - () - Qn() =
{ k
n1
Sn(), k = 0, . . . , n 1}.
, , k > 0.
4. M Qn() -, M k
n1 l
n1 k 6= l
Q(). Pn(S()). ,
n- , .
1. |P (S())| = n|S()|. -
:
ModS()(A) kn1
={M
M P (S()), M |= A kn1
}.
5. , , S() S() S(), P (S())
S()(,) =
n1k=1
ModS()( k
n1 0
)+n1k=1
ModS()(0 k
n1
)
n|S()|.
-
20 (TF) .., ..
2. , , , S() S() S() :
1) 0 6 S()(,) 6 1;2) S()(,) = S()(,);3) S()(,) = 0 ;
4) S()(,) = 1 n1l=1
n1k=1
(Mod() k
n1
Mod() l
n1
)= P (S()),
;5) S()(,) 6 S()(, ) + S()(, );6) 1 2 , S()(1, ) = S()(2, );
3 ( ). S(0) , S() S() S(0) S(1) , S(0) S(1), :
S(0)(,) = S(1)(,).
.
6. IS()()
() ={ S() S()
}
IS()() =
n2
i=0
i
ModS()( l
n1
)
n|S()|,
i : 0 6 i 6 1, i + n1i = 1, k > i, i = 0, . . . , n12 k = 0, . . . , i.
4 ( IS()). , ()1) 0 6 IS()() 6 1;2) IS()() + IS()() = 1;3) IS()( ) > max
{IS()(), IS()()
};
4) IS()( ) 6 min{IS()(), IS()()
};
5) IS()( ) + IS()( ) = IS()() + IS()();6) I3S()( ) = 12
(I3S()() + I
3S()() +
3S()(,)
);
7) I3S()( ) = 12(I3S()() + I
3S()() 3S()(,)
).
, n = 3 .., n = 2, . [1]. n > 3 , .
-
(TF) 21
n = 3, n = 2 (., , [1]). n. n - . n = 2 . - () . - () - , - , , . , - ( ), , , - . .., .. .. (, , ) ( ), -- .
, 07-01-00331.
[1] .., . . - . : - - - , 1999.
[2] ., .. . .: , 1977.
[3] .. .: , 2000.
..
,
: -, -, , -. , .
-
22 (TF) ..
X XL = (x1, . . . , xL) X L. k : XL = XnXkn, +k = L, n = 1, . . . , N N = CkL .
R T : X X R, X X.
, - n, - Xn. X
kn, -
T : X R, Tn = T (X
n) Tn = T (X
kn,X
n), -
Xkn. , . . () ,
Pn
{d(Tn, Tn) >
}6 (), (1)
d : R R R , - d(r, r) r R - r R. , (1 ()) . (1) -, () . () k, T T . (1) , , [5].
, - . - : Pn{(n)} = 1N
Nn=1 (n)
: {1, . . . , N} {0, 1}, XL. , , - .
, - , . . - . - , . .
1. . , , , . . - - : ,
-
(TF) 23
( ), , - [6], . .
2. () , - X - , - XL -, . - , - . - [1]. . , - , - XL. XL
.3. Pn{(Xn,Xkn)} 6 -
, - XL :
EXL Pn{(Xn,Xkn)} = PXL{(X,Xk)} 6 EXL.
, . - [3].
4. , - . - ( - - ). , - -, -. - , .
5. [5] - . , , , - . , - , , . , , . , -
-
24 (TF) ..
. - .
: - - . -. , - .
.. [4]: . . . - , , . - .
.. [2]: . . . , - - -, - .
, 05-01-00877, - .
[1] . . . -, 1980.
[2] . . . .: ,1975.
[3] . . - // / . . . -. .: , 2004. . 13. . 536.
[4] . . / .. . . .: , 1987.
[5] Lugosi G. On concentration-of-measure inequalities. // Machine LearningSummer School, Australian National University, Canberra. 2003.
[6] Vapnik V. Statistical Learning Theory. Wiley, New York, 1998.
-
- (TF) 25
-
.., .., ..
,
- - .
- . - - - . [1] - () . , [2], - . - , , - - [2]. , - , . . - .
- () [3]. P -, q:
P (q, x) = minq
P (q, x), (1)
P (q, x) = supk
min(|(x|k), l(k, q)), (2)
P (q, x) = x = q, =
(1, . . . , n
), x =
(x1, . . . , xn
), xj j-
(), |(x|k) k () x.
|(x|k) -
-
26 (TF) .., .., ..
[4]. -.
k wsk , :
p(k|wsk) = maxs p(k|ws), s = 1, . . . , Sk.
(2) p(x|k) |(x|k). x q, -
(1) ( wsq , ):
P (q, wsq ) = minqP (q, wsq ).
-- - .
[1] .., .., .., .. // .. -. . . 2003. 2. . 12.
[2] .., .., .., .. // .. -. . . 2005. 4. . 3.
[3] .. . . .: ,2000.
[4] .. . - , . .: , 2007.
-
(TF) 27
.., .., ..
[email protected], [email protected]
, . ..,
. - .
- . () : - , - . - , ( ) .
. ( ) . , - : (1) - ; (2) - - ; (3) -. , - .
, - . .
-.
, - , .. .. (). - - , - .
-
28 (TF) .., .., ..
(, , ) - [1]: - 1. - - ( )., - , , - , , , , , . . - .
, , - . - : , - . , - . .. [3] . -. . .
, , , . . [6, 5]. , - [2]. .
. ..-. , .
1 , , , - .. 50- XX .
-
(TF) 29
.
..-. ( 1012) - ( ) -, . - ( - ) . , , -.
1 19 120 ( . [4]). 2 185 ( : 175 + 10).
, , - SIS [7], , - Intel Inc.
- , - - .
, 07-01-00211.
[1] A .., .., .. . .: , 1970. 384 .
[2] .. //- i (). 2005. 3. . 612.
[3] .. . .: , 1967.
[4] .., .., .. // i (). 2006. 2. . 8898.
[5] .. - // . - . ... 2003. 2. . 143147.
[6] . ., .., .. - . - 4.0 www.math.nsc.ru/AP/oteks/Russian/.
[7] SIS: A System for Sequential Circuit Synthesis // Dep. of Electrical Engineeringand Computer Science Univ. of California, Berkeley. 1992.
-
30 (TF) ..
..
, . ..,
[1] -, . .
- , , ,, , , . . , , , - . - , . -, , . , , . - . , , . - , -, , [2, 3].
-RN . - , V RN . [2], . . , .
, [1].
1. RN - (, A, Pr), - RN (), - RN , A - , Pr .
-
(TF) 31
A A () V ==
A RN , Pr(A). -
1 :
2. RN - (, A, P), - , RN , A - , P .
,
, [1]. , , . . - ( ) . - . .
: F1 = (, A, Pr1) F2 == (, A, Pr2), Pr1 Pr2 pr(1)() pr(2)() , - = f + , f RN ( f , ). - , (F1 F2) f . - , :
max (1, 2) min1,2
;
1(x) + 2(x) = 1, x RN ;i(x) > 0, x RN , i = 1, 2;
(1)
idef= 1
RN pr
t(x)i(x) dx, i = 1, 2, pr
t(x) -
, , f .
- ( [5, 6]) . , - -
-
32 (TF) ..
, , - , - . - (. [4]). - - (. [5]), , . ( ). , - . - , . - , .
- , , .
, .
, 05-01-00532-a.
[1] .., .. (- , ) // IX .. . 2006.
[2] .. // . . 1983. . 269, 5. C. 10611064.
[3] Pytev Yu. P. Morphological Image Analysis // Pattern Recognition and ImageAnalysis. 1993. Vol. 3, 1. P. 1928.
[4] .. . - , . , 2007.
[5] .. . : , 1984.
[6] . . - : - . -, 1985.
-
(TF) 33
.., ..
[email protected], [email protected]
,
- , -. - ( ) : - [1], -. -, , - . , , - . [2] . , .
. X, Y X = (xi, yi)i=1 X Y . , yi = y
(xi), y : X Y . - a : X Y , y X.
y : X {0, 1}, y . y x, y(x) = 1.
a(x) = argmax
yY
Tyt=1 w
ty
ty(x),
ty , w
ty
, Ty y. , X
X (X) ={ty(x)
}t=1,TyyY .
y U X
(y, U) =1
|U |
xU
[y(x) 6= [y(x) = y]
].
y X - Xk (y,X,Xk) = (y,Xk) (y,X).
-
34 (TF) .., ..
XL = XnXkn k, +k = L, n N = {1, . . . , CL}.
Q(,XL) - , - Xn XL:
Q(,XL) =
1
|N |
nN
1
|Xn|
Xn
[(,Xn,X
kn) >
].
[0, 1) . , - - [2] .
, : X {0, 1} - XL, (x) = (x) x XL. - (shatter coefficient) (,XL) - XL - , . - XL.
, : L L(,XL) =
Nn=1 X
n.
L L(,XL) = (L,XL) XL.
L L+ 1 , - m XL:m m(,XL) =
{ L : (,XL) = mL
}, m = 0, . . . , L.
XL Dm Dm(,XL) == (m,X
L), m = 0, . . . , L. , L = D0 + +DL. Q Q,m
, m XL:
Q,m(,XL) =
1
|N |
nN
1
|Xn|
Xn
[(,Xn,X
kn) >
][(,XL) = mL
].
1. , XL [0, 1)Q,m(,X
L) 6 DmH(ms1L
), m = 0, . . . , L, (1)
H(ms1L
)=
s1s=s0
CsmCsLm/C
L -
, s0 = max{0,m k} s1 =L (m k)
.
-
(TF) 35
L
crx 690 2.8 108 3.5 104 21 11german 1000 5.2 108 3.1 104 47 38hepatitis 155 5.5 106 1.8 104 58 46horse-colic 300 1.9 106 1.3 104 5 3hypothyroid 3163 5.3 108 2.2 104 43 28liver 345 1.5 107 2.9 104 12 8promoters 106 4.4 109 5.3 104 13 4
1. 7 UCI. 20 = k ; = 0.05.
2. , XL [0, 1)
Q(,XL) 6
L
m=0
DmH(ms1L
).
Dm Dm, - (1) , . . :
Dm = Q,m(,XL)
/H(ms1L
), m = 0, . . . , L.
, Q,m N - N N ( - N ).
, - L = D0+ + DL. , , . - , [3].
:
, - , , : () - ( ) - [3]; () - , {Xn : n N }. - 1.
- L. , -
-
36 (TF) .., ..
hepatitis, 0
20 30 40 50 60 70 80 90 100
0
0.01
0.02
0.03
0.04
0.05
0.06
m
p(m)
horse, 2
40 50 60 70 80 90 100 110 120 130 140
0
0.01
0.02
0.03
0.04
0.05
0.06
m
p(m)
. 1. Dm - p(m) : hepatitis horse.
XL, y (VC-dimension) .
, , . , [4], 101102.
Dm Dm, {Xn : n N }, m - XL. , p(m) = Dm/(D0 + + DL) Dm , , (. 1). .
, 05-01-00877,07-07-00181 - .
[1] Cohen W. W., Singer Y. A simple, fast and effective rule learner // Proc. ofthe 16 National Conference on Artificial Intelligence. 1999. Pp. 335342.
[2] . . - // . . . .: , 2004. . 13. . 536.
[3] . ., . . - // . , 2006. . 281284.
[4] Langford J. Quantitatively tight sample complexity bounds. 2002. CarnegieMellon Thesis.
-
(TF) 37
. ., .., ..
, , - . ..
- () , -, . - . - [3]. - , - - . [4]. - -
M = R(fx, n, s, t), (1)
fx ( );n ; s ; t . fx
fx = f([k], Lxy, hx), (2)
[k] ; Lxy ; hx .
- , - s = f([k], Lxy, hx), - :
argmin[k],Lxy,hx
f([k], Lxy, hx) = min(s). (3)
, - , . - , - , . , -, (1). .
-
38 (TF) .., .., ..
x y, [5], - ai:
d(x, y) =
n
i=1
ai|xi yi|, (4)
d(x, y) x y. , , -
,
d(x, y) =
( n
i=1
|xi yi|p) 1
p
=
( n
i=1
ai|xi yi|) 1
p
= C(p)
n
i=1
ai|xi yi|, (5)
C(p) =n
i=1 ai|xi yi|)1pp ; ai = |xi yi|)p1; p > 0.
, , . . - . , - : , , - [1, 4, 5]. , . - , . -, , -. - . . - , (, ). , . [5].
, , - . - , ,
-
(TF) 39
. -, , , - , , . - . - , , -. , - , , . : (1NN) k (kNN). 1NN , . kNN . k, - , . - , . , - , 1NN kNN , , - .
[1] .., .., .. . - . . :, 2005. 159 .
[2] .., .., .. // . 2005. 2. . 813.
[3] Kapustiy B.O., Rusyn B.P., Tayanov V.A. Tayanov Comparative analysis ofdifferent estimates of Recognition Probability // Journal of Automation andInformation Sciences. 2006. Issue 8. P. 816.
[4] Kapustiy B.O., Rusyn B.P., Tayanov V.A. lassifier optimization in smallsample size condition // Avtomatika i vychislitelnaya tekhnika. 2006. vol.40, Issue 5. P.2532.
[5] Webb R.A. Statistical Pattern recognition. John Wiley & Sons Inc, 2nd ed.,2002.
-
40 (TF) ..
..
-, -
, . , - , , ().
, - [2]. , - - . . , -, ( ). , , , , . - , .
, . . A B, B, , A ( A B ) [2].
X . - f1, . . . , fm, m > 2, f(x) = (f1(x), . . . , fm(x)), X , X. - Sel(X).
X Sel(X) Y = f(X) Rm Sel(Y ) = f(Sel(X)) .
:1) A B - wi (i A), wj (j B) B A j (j B), i (i A); 2) A C - wi (i A), wk (k C) C - A k (k C),
-
(TF) 41
i (i A). A, B, C -. (). , - .
, - , [2].
1 ( ). - y, y Y , y Y y, y / Sel(Y ). 2. Y - Rm. , Y Y . 3. f1, . . . , fm . 4. y, y Rm , y y, > 0 c Rm y ++ c y + c.
. [1] -, - .
.
1. () -, i1 A j B,
wi1wj
>i1j
(1)
i2 A k C, -
wi2wk
>i2k
. (2)
, , .
2. 14 (), (1), (2) i A, j B,k C. Sel(X)
Sel(X) Pg(X) Pf (X),
-
42 (TF) .., ..
Pg(X) - g q = m (|A| + |B| + |C|) ++ 4 |A| |B| |C|
gijk = wjw
kfi(x) + w
iw
kfj(x) + w
jw
i fk(x), i A, j B, k C;
gikj = j
kfi(x) +
i
kfj(x) +
j
i fk(x), i A, j B, k C;
gjki = wj
kfi(x) + w
i
kfj(x) + w
j
i fk(x), i A, j B, k C;
gkji = jw
kfi(x) +
iw
kfj(x) +
jw
i fk(x), i A, j B, k C;
gs = fs, s I \ (A B C).
, 2, Pg(X), - g, Pf (X) Sel(X).
, , 2 , () - ( A - B, B A) [1].
[1] .., .. //. 2006. . 46, 12. . 21782190.
[2] .. : - . 2- . : , 2005. 176 .
.., ..
-, . .
. Fp n, - , - .
n , Fp, Fq=pn ,
-
(TF) 43
- . - , - , ( - ), , . .
Fq [1]. , - , [2]. , - , - , . , .
, - . , - . - , .
[2] -. - , , [3]. F2 - 4 3k 5l, F3 4 2k 5l, Fp (p > 3) 2 2k 3l [4]. - [5, 6].
, n Fq. , n.
n Fq . - . [7], -, [8] - . [9] [10] -
-
44 (TF) .., ..
n Fq, .
, - . , - - n Fq. - [2], . - - n p . - [2] n Fq. - n Fq . [11] , .
- n Fp. , . - p- Fpn , , . . , - . u0 = 1, u1 = 0, u2 = 0, . . . , un1 = 0. - , - , Fp, . An,n Fpn ., .
- , Fpn . Fp,
-
(TF) 45
- .
- - - GAP 4.4.6 Gap Group MAGMA 2.12 Computational Algebra Group.
, , , .
, 07-07-00285, -63.2007.9.
[1] Dickson L. E. Linear Groups with an Exposition of the Galois Field Theory. New York: 1958.
[2] ., . . .: . .1,2. 1988
[3] Gao Sh., Panario D. Foundations of Computational Mathematics. Springer.1997. P.346.
[4] Shparlinski I. Apl. Alg. Eng. Comm. 1993. v.4. P.263.
[5] Gao S., Mullen G. J. Number Theory. 1994. v.49. P.118.
[6] Menezes A., Blake I., Gao X., Mullin R., Vanstone S., Yaghoobian T.Applications of Finite Fields. Kluwer Academic Publisher. 1993.
[7] Butler . Quart. J. Math. Oxford Ser. (2). 1954. v.5. P.102.
[8] Berlekamp E.R. Math. Comp. 1970. v.24. P.713-735.
[9] Rabin M.O. SIAM J. Comp. 9. 1980. P.273.
[10] Shoup V. J. Symb. Comp. 20. 1996. P.363.
[11] .., .. . . .66. 4. 1978. C.197.
-
46 (TF) ..
-
..
, . ..,
- () [1] - [A,] ,
= Af + , (1)
f Rm , A : Rm Rn -, , Rn E = 0 - : Rn Rn, x = E(x, 0), x Rn.
k : Rm Rk k- - Lk Rm
infR:RnLk
supfRm
ER kf2 < .
3. [A,] - (), - f , .. , :
() = max{k, h(k) 6 }, > 0, (2)
4. , (1), : [0,) {1, 2, . . .}, () - R() R -, , - > 0 [1].
- - [2] .
, 05-01-00532-.
[1] .. - . .: M, 2007.
[2] Daubechies I. Ten lectures on Wavelets. SIAM, 1991 (. . ., 2001).
-
(TF) 47
..
,
, -, , .
X , , Y , C - - D = XY . c C : D,B,Pc, B -, Pc[D] . .
f : X Y .
L : Y 2 [0,). :
R(c, f) =
D
L(y, f(x)) dPc[D].
={(xi, yi) D
i = 1, . . . , N} - Pc[D]. - :
R(, f) =1
N
N
i=1
L(yi, f(xi)).
, -, . -
R(,Q) =1
N
N
i=1
L(yi, fQ,i(xi)),
i = \{(xi, yi)} , i- -, Q : {} , fQ, , Q, .
-
48 (TF) ..
R [0, R()]. , . , - - R(), - ().
:
c Pc(R 6 R()
)6 ,
. , -
, . , K
(Fc,R()
),
Fc,R() R(). -, , . .
c K .
c , , - , . -.
(. [1]) , R() == Re
(R()
), -
R, .
, .
. , .
1. R() R, 1 2 -
R(1) > R(1) R(1) > R(2).
-
(TF) 49
- , - . , .
- - .
, 07-01-00331-a, , 7.
[1] .., .. . : , 1974. 415 .
[2] .., . . - . : - ,1999. 211 .
[3] Nedelko V.M. Estimating a Quality of Decision Function by Empirical Risk //LNAI 2734. Machine Learning and Data Mining in Pattern Recognition. ThirdInternational Conference, MLDM 2003, Leipzig. Proceedings. Berlin: Springer-Verlag, 2003. P. 182187.
[4] .. - // .- . 2004. 1. . 4753.
[5] .. // . . . , -11 : , 2003. . 148150.
..
-, -
-, (), - .
- ,
-
50 (TF) ..
. . , , - , - . , - .
, - (., -, [1]). - - - .
- [1]. --, , -, - . - .
, , - , - , , - . , , - , . - : .
- , , . - [5].
-
(TF) 51
, -, (. [2, 4]). - [3]. - , - .
, - - ( ) , (). , , - , 0 1. ( - ), - .
- --, - . - , , . , .
-, - - (), , - 0 1. () . - .
- , , -
-
52 (TF) ..
. - - . , - .
, .
, 05-01-00310.
[1] .. 2- . : , 2005. 176 c.
[2] .. - - // . 2003. . 43. 11. . 16761686.
[3] .. Upper estimate for fuzzy set of nondominated solutions // FuzzySets and Systems. 1994. Vol. 67. P. 303315.
[4] .., .. - // - . 2006, 1. . 2133.
[5] Zadeh L.A. Fuzzy sets // Informat. Control. 1965. Vol. 8. P. 338353.
..
, . ..,
- . , - , , , , , -, , - [1].
- , X = {x1, . . . , xn}, . = x1, . . . , = xn .
-
pj , P( = xj), j = 1, . . .
-
(TF) 53
pi1 ,pi2 , . . . ,pin - = x1, . . . , = xn, - i- , i = 1, . . . ,m. i- , (i(1), . . . , i(n)) , (i1, . . . , in), ( )
1 = pi1 > pi2 > . . . > pin , i = 1, . . . ,m. (1)
, .
r(i, t) i- s-
r(i, t) =
( n
k=1
(i(k) t(k))2) 1
2
(2)
(2) t1, . . . , tn , -, , m . t , (
t (1), . . . ,
t (n)),
m , -
m
i=1
r(i, t ) = min
()
m
i=1
r(i, ), (3)
() : {1, . . . , n} {1, . . . , n}.
m
i=1
r2(i, ) =
m
i=1
r2(i, ) +mr2(, ), (4)
(k) =1
m
m
i=1
i(k), k = 1, . . . , n,
(3) t , - 1 : r2(, t ) = min
r2(, ). -
((1)) 6 . . . 6 ((n)), , t = 1.
t -, (4) , .
1(1), . . . , (n), , 1, . . . , n.
-
54 (TF) ..
, {1, . . . , m} m ,m (n!)m , Pr({}) = (n!)m, m, P(m).
(m,P(m),Pr) m , .
s() ,mi=1
r2(i, ), m,
S = {s(), m} , {s1, . . . , sL} prl , Pr({
m, s() = sl}), l = 1, . . . , L,
: pr1 > . . . > prL. H0 , - , , H0 - 6 , > 0, S , {sl(), sl()+1, . . . , sL}, l() = min{l, sl S,prl + . . .+ prL 6 }.
, H0
, mi=1
r2(i, ) S , t .
, 05-01-00532-.
[1] .. . - , . .: , 2007.
-
..
, . ..,
- - -, , - .
(,P(),Pr), = {1, 2, . . .}, P() , pri
def= Pr({i}), i = 1, 2, . . . ,
Pr() : P() [0, 1], , .
-
- (TF) 55
-
: (n,P(n),Pr(n)), Pr(n) = Pr . . . Pr, (n, P(n),Pr
(n)1,...,n), Pr
(n)1,...,n
= Pr1 . . . Prn, n = 1, 2, . . .
( ) , , - .
n (n)1 ,
(n)2 , . . . 1, 2, . . . , n = 1, 2, . . .
:
1. pr1,pr2, . . .
2. - (,P(),P), P() : P() [0, 1] Prj- -, j = 1, 2, . . . , [1].
3. (,P(),P) - A - , , P Prj A , j = 1, 2, . . .[1]. 1 -
pri1 > . . . > pris , (1)
, 1 , (0, 1), , pri1 , . . . ,pris , 1, s == 2, 3, . . . (..) .
Pr- - P, , (1) ( - ) -
pi1 > . . . > pis > . . . pidef= P({i}),
i = 1, 2, . . . , Pr-1 P:
pik > pik+1 fikdef= pri1 + . . .+ prik1 + 2prik > 1;
pik = pik+1 fik < 1, k = 1, 2, . . . ,
P - [1]
{fi1 > 1 fi1 < 1}, . . . , {fis > 1 fis < 1}, . . . (2)1 Pr : fik 6= 1, k = 1, 2, . . .
-
56 (TF) ..
2 (2) - s = 1, 2, . . . .. .
, 3, , 2 - , , P(). P() - A, , - , .
3. - A, - A .
- , , - Prj-, j = 1, 2, . . . , . , , .. - , - [2].
, 05-01-00532-.
[1] .. . - , .: , 2007.
[2] .. - // IX . . - , 2007.
..
,
: - . -
-
(TF) 57
. - , .
- n.
- SVM [1, 2], - . - : () , SVM - . : xn+k = gk (x1, . . . , xn), k = 1, 2, . . . - . - .
- . , - .
-. - . .
1. n- - n- .
- , - , - .
, g (x1, . . . , xn), , - .
-
58 (TF) ..
1.5 2 2.5 3 3.5 4 4.5
0
0.5
1
1.5
2
Geometrical complexity
Log
( F
unct
ion
com
plex
ity)
. 1. ( 50 ).
- F =
{fi : R
ki R i = 1, . . . ,m
}.
, fi F - ci, . - - .
, - . - , . - .
. - .
, - , -. , , - SVM. , , .
-
(TF) 59
-
- - , .
SVM SMO [3].
. -, -. -, , -
, fi (x1, x2) = xi11 x
i22 , -
C(1) + C(2),
C() =
, = 1, 2, 3, . . .;
+ 1, = , = 1, 2, 3, . . .;, = 1 , = 2, 3, . . .;
+ 1, = 1 , = 2, 3, . . ..
. 1. - . .
, 06-07-89315-.
[1] Vapnik V. Estimation of Dependences Based on Empirical Data SpringerVerlag, 1982.
[2] Burges C. J. C. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery. Vol. 2, No. 2. 1998.
[3] Platt J. C. Sequential minimal optimization: A fast algorithm fo trainingsupport vector machines. Technical Report MSRTR9814, MicrosoftResearch, 1998.
-
60 (TF) ..
..
,
- ().
, [1]. -
. , .
I0 Sq Z = (I0, Sq). , - () B = {B1, . . . , Bn}. - A, {Bk}. Bk Bk (Z) = uvk ql; C - . - A , . . , .
,
A =
( n
k=1
Bxkk
) C (c1, c2) ,
Z, . [2]. xk, k = 1, . . . , n, , .
C , - A - Bk.
{Su : u = 1, . . . , q} {Kv : v = 1, . . . , l} 2 :
M0 = {(u, v) : Su 6 Kv}; M1 = {(u, v) : Su Kv}.
[2] , -
-
(TF) 61
y = {y1, . . . , yn}:
(u, v) M0 : uv (y) 6 c1; (u, v) M1 : uv (y) > c2;max
k=1,...,nyk min;
: yk = exk , uvk = lnuvk ,
uv (y) =n
k=1 yuvkk .
[3], , 0 6 yk 6 ql, k = 1, . . . , n.
R ={y U, max
k=1,...,nyk min
},
U =
y
uv(y) 6 c1, (u, v) M0;uv(y) > c2, (u, v) M1;0 6 yk 6 ql, k = 1, n
.
[2] , - RI = {y UI , yi1 min}, UI = {y U, yi1 = yi2 = . . . = yim}, - I = {i1, . . . , im} {1, . . . , n}. , RI - [4].
- - , [5]. , [6].
RI , - , . - .
- . B == {Bk}, Z. - uv(y), . - B B, - R (B) F (B), Z.
-
62 (TF) ..
, B B - uv(y) 1, (u, v) M1 - uv (y) = c2; 0, (u, v) M0 uv (y) = c1. - B M1 M (Bk), k = 1, . . . , n. , B, , , B , 1.
, .
. , 05-01-00718,
06-07-89299, 07-07-00181, - -5833.2006.1.
[1] .., .. , - //. . . .. 1979. . 19, 3. . 726738.
[2] .. // . . . -. . 2007. . 47, 8. . 14261430.
[3] .. - . c. . .-. ,.: , 1992.
[4] .. . .: , 1988.
[5] .., .., . - - //. . . . . 2004. . 44, 9. . 15641573.
[6] Coleman T. F., Li Y. A Reflective Newton Method for Minimizing a QuadraticFunction Subject to Bounds on some of the Variables // SIAM Journal onOptimization. 1996. Vol. 6, 4. Pp. 10401058.
-
(TF) 63
.., .., ..
,
( , -, , , ) - . , - ( ) (), - -. , , - . , , - [14].
, - ( , - , ), - -. , , ., , - (, , , ). k- -, -, . - . - . , , : - .
- - . . m- m -, (m1)- , m-
-
64 (TF) ..
. m- (m1)- . - m- (m1)- m-. - .
-, - . - . - .
[1] .. . : . .: , 1989.
[2] ., . . .: , 1976. 511 .
[3] . . . .: . ,1972. 206 .
[4] . . . -: - , 1999.
. .
,
- : - ?.
.. - -, [4]. - - ... - .
- . - . - .
-
(TF) 65
, ( - ). . - .
, - ( ) - , .. [5, 6].
. , , . , . , , , .
- - K(n,m) . - f(m) n , K(n,m) = f(m), n f(m). . - , - . - .
1. g(n) n0, , K(g(n0),m) = K(n0,m). , g(n0) n0 , n0 - g(n). g(n) , .
2. - (n) - (n) , K((n),m) = K((n),m), (n) , (n) -, (n) .
, 1, .. -, , 2, [5, 6]. .
-, , .
-
66 (TF) ..
, - . , - [6].
[13], - [7, 8].
, - . [2].
(n) = K(n,m), ,K(n,m) .
{Ar}, r - , Ar :
1) r Ar;2) r1, . . . , rn Ar,
ni=1 ri
{n, (n)} 6= n Ar (n) Ar;
3) Ar .
Ar . -, - Ar, , - , , .
, , - . , . -. , , - , .
[1] . . : , 1977.
[2] . 1. : , 1966.
[3] Ershov Y. Theory of enumerations, Part II // Z. Math. Log. und Grundl.Math. 1975. . 21. Pp. 473584.
[4] .. // 1965. 1. . 17.
[5] .. // 1963. 2. . 429.
-
() (TF) 67
[6] . // - 1964. 4. . 531.
[7] .. // . . . . : , 2003. . 699.
[8] Trofimov O.E. Chaos and Recursive Denumerrability // Int. conf. on modelling& simulation (ICMS04-Spain), Valladolid, 2004. Pp. 1933194.
()
.., ..
, . ..,
() , - , , - , - () , , , () - [6, 5].
(Y,P(Y ), P) , - Y , P(Y ) - Y , , P: P(Y ) [0, 1] ().
P() f(y) , P({y}) == P( = y), y Y, {y} Y, ,P(A) , P( A) = sup
yAf(y), A P(Y ).
f : Y [0, 1] (Y,P(Y ), P) : Y (Y,P(Y ),P()).
(U,P(U),Pl()) -, U , P(U) U (), Pl() : P(U) [0, 1]- . P(), Pl() gu() : U [0, 1] u : (U,P(U),Pl()) U. 1 (, [5]). () -, X, , q(, u) (-) (, u) u q : Y U X.
-
68 (TF) .., ..
x(p) , Pl(P ( = x) = p) = sup{gu(u)
u U, fu(x) = p},
x X, p [0, 1], - , , , . - x(p) -, p = x X.
[6, 7] , - . [2, 3, 4] - , , - .
, , , - , , - , - .
, 05-01-00532-.
[1] .. . - // 1- -- -. 2005. C. 482492.
[2] .. // -13( ). 2007. C. 5254.
[3] .. - - - // -13 ( ). 2007. C. 5456.
[4] .. - // IX . . - . 2007.
[5] .. // - . 2004. 8, . 14. C. 147310.
[6] .., .. - // -12. 2005. C. 202206.
[7] .. -. . . .-. . 2006.
-
(TF) 69
..
,
[1, 2, 3] . -, - [4, 5, 6] . - , , , .
(, ), , , .
, - .
- , . - , ( ). , - , , , - , , - .
-, , - - . , . , -
-
70 (TF) ..
.
- , , , - , . - , -.
- . , - , , , - , , - . , .
, - - - . , , -. , , , . - , , , .
, , - , .
, 07-07-00711.
[1] .. - // . 1987. 2. . 3035.
-
- (TF) 71
[2] .. - // . 1987. 3. . 106109.
[3] .. - // . 1987. 4. . 7377.
[4] .., .., .. // . 2007. . 416, 4.
[5] .. - // . . -11, , 2006. . 210211.
[6] .. - // . 1993. 1. . 155159.
-
- ..
,
- , - -, - . - , , -, ; ( ) [2], - ( ), ( ) . .
- . - - , - , - [1] - [4].
, - . , -, -. , -
-
72 (TF) ..
(, , ), () - . , , , - - . ,, , , , , .
, , . ( ) - . , , , . , - .
, - -- . [3] - Ii , If .
, - Ii Ii (Ii) -, , - . - - . , , - . .
, . , , , , -.
, - , -. , , -
-
(TF) 73
(Ii), . - (Ii) - , - . - -.
, -, , Ii - - Ii.
, 07-07-00711 -5266.2007.9.
[1] .. - // . . 1978. . 33. . 568.
[2] . ., .., .. - // . . -2006, 2006, . 9697.
[3] .. // . 1987. 2. C. 3035, 1987. 3. C. 106109, 1987. 4. C. 7377.
[4] .., .. - - // . 2005. Vol 45. 2. . 344353.
. .
, -
( [1]) () [1]. - .
-
74 (TF) ..
, - - , , , , . , - -, , .
, [1, 2, 3], - . [4] - . - 5 [5]. 6 - . .
, - [2, 5].
, - , . -, -. , , , - .
N = {N1, . . . , Nn} ; Z == {z1, . . . , zm} ; Li , - Ni, i {1, . . . , n}; T , - Ni, Ni N , i {1, . . . , n}, xj zj Z, j {1, . . . ,m}; n0 - ; T0 T; N0 , T0, N0 N .
wrj (wgj ) zj Z,
j {1, . . . ,m}, [2]( wgj =
kjn0
, kj - xj T0).
Ni, i {1, . . . , n} :1) W ri =
jLi
wrj ;
2) W gi =
jLiwgj ;
-
(TF) 75
3) W i =
jLiwj , w
j
zj Z, j {1, . . . ,m};4) W i =
jLi
wj , wj (), -
() j- .
.
N , T; Z, N ; : wrj w
gj , -
wj () wj , j {1, . . . ,m}
j- ; W rj Wgj , W
j ,
() W j , i {1, . . . , n} . T T0, -
n0 , - N0 :
1) N0 ;
2) N0 ;
3) N0 W r0 =
iN0
W ri ;
4) N0 - W g0 =
iN0
W gi ;
5) N0 -;
6) N0 ().
, 4) - [1], , N0, - .
1. W g0 N0,
n0, Kn0 - T0.
. W g0 =n0j=1
wgi =n0j=1
kjn0
=Kn0n0
, -
, W g0 n0 = Kn0 , .
-
76 (TF) ..
4) : - N0 .
.
[5] (--, , ) - 4) -. - [6].
, 07-01-00452 , 06-06-12603.
[1] .., .. - // : 3- . .2. : ./ . ... .: , 1990. . 149191.
[2] .. - // - : . 3- . . . .,: - , 2000. . 163168.
[3] Naidenova R.A., Plaksin M.V., Shagalov V. L. Inductive inferring all goodclassification test // --. . . . . . 1. , 1995. . 7984.
[4] .. - // - . . , 2002. . 100102.
[5] .., .., .., .. // - . . -2005. . 1. .: , 2005. . 256262.
[6] Yankovskaya A.E., Gedike A. I., Ametov R.V., Bleikher A.M. IMSLOG-2002 Software Tool for Supporting Information Technologies of Test PatternRecognition // Pattern Recognition and Image Analysis. 2003. Vol. 13,4. P. 650657.
-
: MM (Methods and Models)
() .
.
.
.
.
.
.
.
, .
.
-
78 (MM)
-
(MM) 79
.., ..
,
, [1, 2]. [1] - . - , [2], . - . , - , , -.
- n0 [3].
00 = {X0,1, . . . ,X0,n0} - i, i = 1, 2. - i0 = {Xi,1, . . . ,Xi,ni}, - , , i f(y, i), i = 1, 2.
.(C1). i -
f(y, i) = h(y) exp {i T (y) + V (i)} , y G R, i R.
h(y), T (y) , V () .
(C2). Sni =ni
j=1 T (Xi,j) i i0, gm(t, i) -
mj=1 T (Xi,j).
(C3). Yi = T (i), i = 1, 2, --
fYi(t;i, ) =t1
i () et/i , t > 0, i =
1
i> 0, > 0.
-
80 (MM) .., ..
(C4). Bi ={n0
j=1
(T (X0,j) i
) > nii},
i (0, 14
),
limni
ni P (Bi) = 0, i = 1, 2.
, , 1 > 2, . [2, 3], .
1. (C1), (C3) - 00 :
00 1, q(t) = ln1gn0(t, 1)
2gn0(t, 2)> 0, (1)
1, 2 . (1)
J1 = {t : q(t) < 0} = {t : t < c} ,J2 = {t : t > c} ,
c = ln 12 n0 ln
12
12
11.
(1) -
R = 1 P1 {00 2}+ 2 P2 {00 1} =2
j=1
j
Jj
gn0 (t, j) dt.
- 00:
00 1, q(t | Sn1 , Sn2) = ln1gn0(t | Sn1)2gn0(t | Sn2)
> 0, (2)
gn0(t|Snj ) gn0(t, j), , [4], -.
(2) -
J1(Sn1 , Sn2) = {t : q(t | Sn1 , Sn2) < 0} ,J2(Sn1 , Sn2) = {t : q(t | Sn1 , Sn2) > 0} .
(2), [2],
R = R(Sn1 , Sn2) =2
j=1
j
Jj(Sn1 ,Sn2 )
gn0(t | Snj )dt. (3)
5 [2], , (3).
-
(MM) 81
2. (C1)(C4), c > 0, n1 6 n2, - (2) c,
[R]2=
2
j=1
[j j ]2
nj, [j ]
2=
[c gn0(c | Snj )
]2
, c > 0. (4)
n1 - (RR)/R .
1. - c, . - [R]
2 5 [2], - (4). - .
, [1, 2] , - .
, 05-01-00229.
[1] .., .. - // . . -, 2006. . 411.
[2] .. , , // . . -, 2007. . 5971.
[3] .., .. -. . -, 1987. 92 .
[4] . ., .. . .: , 1989. 440 .
-
82 (MM) .., .., ..
.., .., ..
[email protected], {avezhnevets, vvp}@graphics.cs.msu.ru
, . . .
. , (boosting) , - . [1] . - - .
. - T = {(xi, yi)}Ni=1, xi X ,yi {1, 1} . (xi, yi) - P (x, y). :
RN (F ) =1
N
N
i=1
C (yi, F (xi)) min,
F (x) , C : R R R .
, :
{(xi, yi) T
P (yi |xi) > 0.5 > P (yi |xi)}.
. - , , , .
- .
-
-
-
(MM) 83
1. .
: T = {(xi, yi)}Ni=1; K;
: p(1 |xi), i = 1, . . . , N ;
1: k = 1, . . . ,K2:
T 1k T 2k T 3k = T : T ik T jk = , i 6= j;3: T 1k , -
Fk;4: A, B Fk ( -
) T 2k - [3];
5:
: pk(1 |xi) =1
1 + exp (AFk (xi) +B);
6: - : p(1 |xi) = 1K
Kk=1 p
k(1 |xi);
:
RN (F ) =1
N
N
i=1
(P (1 |xi)C
(1, F (xi)
)+ P (1 |xi)C
(1, F (xi)
)).
, DRN (F ) < DRN (F ), R
N (F )
, RN (F ). . -
T ={(xi, yi)
}Ni=1
{(xi,yi)
}Ni=1
[2]: D1(i) =
1N , i = 1, . . . , N , D
1(i) = p(yi |xi), i = 1, . . . , 2N .
- . , - . 1.
p(1 |xi) . -, p(yi |xi) < 0, 5 < p(yi |xi),
-
84 (MM) .., .., ..
. . .
Breast 3.73 4.6 3.65 4.58Australian 13.06 15.2 13.8 12.75German 24.66 25.72 25.35 24.8Heart 18.34 21.41 18.36 16.67Pima 24.03 25.58 23.99 23.31Spam 6.49 6.19 6.02 6.06Vote 4.51 4.75 4.61 4.37
1. (%) , ,
. , -, .
- p(1 |xi) : D1(i) == p(yi |xi), i = 1, . . . , 2N .
[2] 1 - 100. 50 2 : 50 2 , .
1 - UCI. . , - . , (). , - ( ).
, .
[1] Friedman J. Greedy function approximation: a gradient boosting machine //Annals of Statistics. 2001. Vol. 29, 5. Pp. 11891232
[2] Schapire R., Singer Y. Improved boosting algorithms using confidence-ratedpredictions // Machine Learning. 1999. Vol. 37, 3. Pp. 297336
[3] Niculescu-Mizil A., Caruana R. Obtaining calibrated probabilities fromboosting // 21st Conf. on Uncertainty in Artificial Intelligence, Edinburgh,Scotland, 2005.
-
(MM) 85
.., ..
, . ..
, - , -, - . - -, CBFAClassification Based FunctionApproximation.
Xn , F : Xn R - , k x1, . . . , xk, . . - T =
{(xm, ym)
ym = F (xm), m = 1, . . . , k}.
CBFA -.
1. M , < ym < M m == 1, . . . , k. [,M ] F Xn.
2. [,M ] l j = [j1, j), j = 1, . . . , l,, {x1, . . . , xk}. 0 = ; l = M .
3. j - Fj ym , ym j . - F .
4. : Xn {1, . . . , l}, - j , y j . F - [,M ]. T T0 =
{(xm, (xm))
m = 1, . . . , k}.
.5. -
T0 m D : Xn {1, . . . , l}, j = D(x) x Xn. j Fj - x. , -
-
86 (MM) .., ..
(- (x)) , P
{D(x) 6= (x)
}< .
1 , (x), - (x) = D(x) x Xn. D [j1, j), F (x) . P{F (x) [j1, j)
}> 1 , P
{} .
-.
1. D . x Xn, D j, F , 1 , [j1, j), j = 1, . . . , l.
1. D x Xn Fj , 1 , F (x) , = max
{Fj j1, j Fj
}.
, - . CBFA - . CBFA - - - [1, 2]. CBFA . CBFA , , ( ).
[1] .. - // . 2000. 2. . 912.
[2] .. // . 2006. 2. . 1013.
-
-- (MM) 87
--
.., .., ..
[email protected], [email protected],
, . ,
-; , [1, 3].
- . -- [2]. , , , [2] , - , - , - [4]. , -- - , - .
- [1] , . fc(x) : X R , - c Y = [1, . . . , C]. , - c (, ). --
F (x) = argmaxcY
fc(x).
- . :
P (c|x) P (c|x) = 11 + exp(Afc(x) +B)
,
-
88 (MM) .., .., ..
A B . :
F (x) = argmaxcY
P (c|x).
, .
: --, - (ECC) -- . 3, . ( ) (. 1): ECC - , , - . 40 , - ( - ).
, -- - , , - ECC.
[1] J. Platt Probabilistic outputs for support vector machines and comparisonto regularized likelihood methods. // Advances in Large Margin Classifiers,1999. pp. 6174.
[2] R. Rifkin, A. Klautau. In Defense of One-Vs-All Classification. // The Journalof Machine Learning Research, 2004. pp. 101141.
[3] A. Niculescu-Mizil and R. Caruana Predicting good probabilities with super-vised learning. // Proceedings of the 22nd international conference on Machinelearning, 2005. pp. 625632
[4] Chun-Nan Hsu and Yu-Shi Lin Boosting Multiclass Learning with RepeatingCodes // Journal of Artificial Intelligence Research, 2006. pp. 263286.
-
-- (MM) 89
. 1. .
-
90 (MM) .., .., .., ..
.., .., .., ..
,
- . , - , , - , . , - [1, 2]. , ,. . . , .
y : X R - X = {xi}i=1 X yi = y(xi). - p fj : X R, j = 1, . . . , p. aJ (x) =
jJ wjfj(x),
J {1, . . . , n} , wj ,, wj 1. X J = J(X) , - aJ (x) y(x) X . U X E(J, U) = 1|U |
uU |aJ(u) y(u)|,
(J, U) = 1|U |
uU[|aJ (u) y(u)| >
], .
, . .
- . , , - .
- . , ,
-
(MM) 91
. ; - , .
, - - . , , - .
, - , y(x) =
pj=1 fj(x).
, , (, , , . .) , ; - . , .
(Xn,Xkn) = (Jn,X
kn) (Jn,Xn) -
Jn = J(Xn) n- XL -
Xn Xkn, n N {1, . . . , CL}
. - .
.
M1. t . - X E({j},X), j = 1, . . . , p, J(X) t -. , .
M2. t T . - , T - J , |J | = t, E(J,X) . CtT R, R .
M3. . J - . - j, E(J {j},X) .
. ( )p = 89, ( ) L = 35, - fj(xi) , i- j- .
-
92 (MM) .., .., .., ..
5 10 15 20 250
0.05
0.1
0.15
0.2
0.25
0.3
t
Qua
lity
Training M1Test M1Training M2Test M2Training M3Test M3
. 1. - t M1, M2, M3. - : = 17, k = 18, T = 35, R = 1300; |N | = 30 .
, , , .
-, . M1 -. M2, -. M2 T t = 2 3, 23 - T . M3 .
, 05-01-00877 - - .
[1] Miller A. Subset selection in regression. Chapman & Hall/CRC, 2002.
[2] Zhang P. Inference after variable selection in linear regression models //Biometrika. 1992. No. 79. Pp. 741746.
-
(MM) 93
.., ..
[email protected], [email protected]
, ,
, , - (RVM), - , - [2]. RVM - . , , [1]. , - RVM - , , . , RVM , .
, - . .
, {(xi, ti)}ni=1 == (X , T ), d- x Rd - , t {1,+1}. -
y(x,w) =
M
i=1
wii(x) = w, (x), (1)
w , , (x) = {i(x)}Ni=1 ( ).
L(T |X ,w) = n
i=1
log(1 + exp(tiy(xi,w))
). (2)
P (w|). wMP = argmax
wexp(L(T |X ,w))P (w|). RVM -
-
94 (MM) .., ..
wi N (0, 1i ) i . :
P (T |) =
P (T |X ,w)P (w|)dw max
. (3)
(3) . RVM - ( ) , -. , -, , .
, - - . (3)
P (T |) P (T |X ,wML)
exp
(1
2(w wML)TH(w wML)
)P (w|)dw,
wML (H)1 - .
, H: u = Qw, H = QTQ, == diag(1, . . . , M ), {i}Mi=1 H. - (2) , H 6 0 - {i}Mi=1 . hi = i > 0. u ,
P (u|) =M
i=1
P (ui|i).
- (3)
P (T |) P (T |X ,uML)M
i=1
exp
(hi
2(ui uML,i)2
)P (ui|i)dui
fi(hi,uML,i,i)
. (4)
-
(MM) 95
. i, . (REVM).
ui N (0, 1i ) fi(hi, uML,i, i) - (4) :
fi(hi, uML,i, i) =
i
hi + iexp
(hiiu
2ML,i
2(hi + i)
). (5)
hi uML,i (5), i, [0,+). (5) i , - i:
i =
{hi
hiu2ML,i1, hiu2ML,i > 1;
+, . 1. 1(x) 2(x) == A1(x), A M M . (1), (X , T ) y(x;,X , T ).
yRVM (x;1,X , T ) 6 yRVM (x;2,X , T );yREVM (x;1,X , T ) yREVM (x;2,X , T ).
, RVM,REVM ( u).
, 06-01-08045,05-07-90333, 06-01-00492, 07-01-00211.
[1] Williams P.M. Bayesian regularization and pruning using a Laplace prior //Neural Computation. 1995. V. 7, 1. Pp. 117143.
[2] Tipping M.E. Sparse Bayesian Learning and the Relevance Vector Machine //Journal of Machine Learning Research. 2001. Vol. 1, 5. Pp. 211244.
-
96 (MM) .., ..
.., ..
[email protected], [email protected]
, ,
(generalized linear models) . , , . , , , .
- (X,T ) = {xi, ti}mi=1, x Rd, t {1, 1}.
, , -
L(T |X,w) = m
i=1
log
(1 + exp
(ti
N
j=1
wjj(xi)
)), (1)
{j(x)}Nj=1 , -. (1) -, w . - ,
Rl(w, ) = wIw = N
j=1
w2j .
, - -. - , , , - ( ) , .
-
(MM) 97
[3] -
Rr(w,) = ww = N
j=1
jw2j ,
. , - . j
= argmaxE() = argmax
exp (L(T |X,w) +Rr(w,)) dw.
Ri(w) = wAw, A = A, A > 0.
A - E(A).
wML = argmaxL(T |X,w), H = L(T |X,w)|w=wML > 0. M = H(H +A)1A. [1],
logE(A)
A= 0.5
[M1 wMLwML
].
A1 = wMLw
ML H1.
A , - . - U , D = UA1U . D1 + ( , ) , - A w, , u, - d1
-
98 (MM) .., ..
SVM RVM LogReg IREVM VRVM
Bupa 28.46 33.33 57.97 29.68 30.78Heart 19.33 18.15 22.44 18.00 17.56Hepatitis 17.55 14.32 19.48 16.26 13.03Votes 4.78 5.56 5.98 4.92 5.61WPBC 21.72 23.64 23.84 21.11 24.04Laryngeal1 17.75 16.81 19.44 17.28 17.37Weaning 10.99 15.70 13.31 12.72 13.64
17.00 21.00 32.00 14.00 21.00
1 2 3 4 5
1. ( %).
D1. , - wMP = u = argmax
(L(T |X, u+ d12)
).
, , - -, - .
1 , UCI [4]. (RVM) (VRVM) [2] , (SVM) (LogReg). , - - .
, 06-01-08045,05-07-90333, 06-01-00492 07-01-00211.
[1] Kropotov D.A., Vetrov D. P. Optimal Bayesian Linear Classifier with ArbitraryGaussian Regularizer // 7th Open German-Russian Workshop on PatternRecognition and Image Understanding (OGRW2007), Ettlingen, 2007.
[2] Bishop C.M., Tipping M.E. Variational Relevance Vector Machines //Uncertainty in artificial intelligence (UAI-2000), 2000 P. 4653.
[3] Tipping M.E. Sparse Bayesian Learning and the Relevance Vector Machine //Journal of Machine Learning Research. 2001. Vol. 1, 5. P. 211244.
[4] Asuncion A., Newman D. J. UCI (Machine Learning Repository) 2007. www.ics.uci.edu/~mlearn/MLRepository.html.
-
(MM) 99
.., .., ..
[email protected], [email protected], [email protected]
, , ,
D = (X,T ) = {xi, ti}mi=1, x Rd,t {1, 1}. . - , , ( ) . , , -, . , -, , .
, , - , , - :
p(t|x) = 11 + exp(ty(x)) ,
t Y = {1; 1}, y(x) = w0 +w1x1 + +wdxd. :
L(Ym|Xm,w) =n
i=1
ln(1 + exp(tiy(xi))
). (1)
w (1). : - wi . (1) [1]:
F = L(Ym|Xm,w) + R(w), (2)
R(w) =d
i=1 |wi|.
-
100 (MM) .., .., ..
, (2) -. - -. -, - , p() 1/ - . p(w|) -
p(w|) =(
2
)Nexp
(R(w)
)=
N
i=1
2exp
(|wi|
),
N . -
p(w) ==
p(w|)p()d. -
:Q = L(Ym|Xm,w) +N lnR(w). (3)
(BLogReg), - , 2006 [3]. - . - . , (3) , , - . [2] - .
(3) , , , - , - . - (3) :
Q = L(T |X ,w) + N logR(w); (4)
N = nn
i=1
exp
( w
2i
22
). (5)
> 0 . HML,
-
(MM) 101
(1). , - (4) M = {w HML, |wi| > > 0,i = 1, . . . , n}. (4) M - -, .
1. , (5). - (4) ,
1
2= o(2 ln1 ).
UCI [4] , . - , - -, BLogReg. - 10 25 , 100 820 .
[1] Williams, P.M. Bayesian regularization and pruning using a Laplace prior. //Neural Computation. 1995. Vol. 7. P. 117143.
[2] Figueiredo M. Adaptive sparseness for supervised learning. // IEEETransactions on Pattern Analysis and Machine Intelligence. 2007. Vol. 25. P. 11501159.
[3] Cawley G.C., Talbot N. L. Gene selection in cancer classiffication using sparselogistic regression with bayesian regularization. // Bioinformatics. 2007. Vol. 22. P. 23482355.
[4] Asuncion A., Newman D. J. UCI Machine Learning Repository 2007. www.ics.uci.edu/~mlearn/MLRepository.html.
-
102 (MM) .., .., ..
Expectation Propagation
.., .., ..
[email protected], [email protected], [email protected]
, ,
. D = {(xi, ti)}Ni=1, xi Rd, ti {1, 1}. y(x) = sign(wT(x)), w RM , (x) = [1(x), . . . , M (x)] . w -:
p(t|X,w)p(w|) = p(w|)N
i=1
p(ti|xi,w) = p(w|)N
i=1
(tiwT(xi); 0, 1),
t = {ti}Ni=1, X = {xi}Ni=1, (y;m, s2) = 12sym
exp( x22s2
)dx -
(-), p(w|) N (0, A1), A = diag(1, . . . , M ). - [1]:
p(t|X,) =
p(t|X,w)p(w|)dw max
.
, , -. . expectation propagation () [2].
Expectation Propagation
EP , - . , - .
p(w|X, t, ) p(w|)N
i=1
p(ti|xi,w) p(w|)N
i=1
gi(w)
p(w|)N
i=1
gi(w) = q(w).
, - - - (
-
EP (MM) 103
1. Expectation Propagation.
: gi(x), i = 1, . . . , N ;: gi(x), i = 1, . . . , N ;1: : gi = 1; vi = ; mi = 0; si = 1; q(w) = p(w|);2: // (mi, vi, si) 3: i = 1, . . . , N4: (a) gi q(w).
q\i(w) q(w)/gi N (m\iw ,V\iw);V
\iw = Vw +
Vwi(Vwi)
viiVwi; m\iw = mw + (V
\iwi)v
1i (
imw mi);
5: (b) p(w) gi(w)q\i(w). q(w), KL(p(w)q(w));mw = m
\iw +V
\iwii; Vw = V
\iw + (V
\iwi)
i(
imw+i)
iV\iw i+1
(V\iwi)
;
Zi =wgi(w)q
\i(w)dw = (zi),
zi =(m\i
w)i
iV\iw i+1
; i = 1iV
\iw i+1
N (zi;0,1)(zi)
;
6: (c) gi = Ziq(w)q\i(w)
, :
vi = iV
\iwi
(1
i(imw+i) 1
)+ 1i(imw+i) ;
mi = im
\iw + (vi +
iV
\iwi)i;
si = (zi)
1 + v1i
iV
\iwi exp
(12iV
\iwi+1
im\iw+i
i
);
7: :
B = (mw)(Vw)
1(mw)
im2ivi
;
p(t|X,) N
i=1 gi(w)dw =|Vw|1/2
(
j j)1/2 exp(B/2)
i si;
KL-):
KL(pq) =
q(x) logp(x)
q(x)dx.
gi(w) gi(w) =
= si exp( 1
2v2i(tiw
T(xi)mi)2), (mi, vi, si).
, q(w) , p(w|X, t, ) q(w). 1 EP( tii i, (z; 0, 1) (z)).
-
104 (MM) .., .., ..
EP
, - , -
p(t|X,w) =N
i=1
(tiwT(xi)) =
N
i=1
1
1 + exp(tiwT(xi))).
wMP p(w|X, t, ). i = 1, . . . , N (y) yiMP = tiw
TMP(xi) - (y;mi, si
2), - p(w|t, ) =
Ni=1 (y;mi, si
2). -mi si2 - yiMP
si2 =
12S(1 S)
exp
(121(S; 0, 1)
), mi = y
iMP 1(S; 0, 1)si,
S = (yiMP ), 1(y; 0, 1), -1.
p(w|t, ) =
Ni=1 (y;mi, si
2). , -
, - ( ), ( - ) , - . , - .
, 07-01-00211,06-01-08045, 05-07-90333.
[1] Tipping M. Sparse Bayesian Learning and the Relevance Vector Machine //Journal of Machine Learning Research. 2001. Vol. 1, 5. P. 211244.
[2] Qi Y.A., Minka T.P., Picard R.W., Ghahramani Z. Predictive AutomaticRelevance Determination by Expectation Propagation // 21-st InternationalConference on Machine Learning, Banff, Canada, 2004.
1, , MATLAB norminv.
-
(MM) 105
.., ..
,
() .. 70- [1]. - , direct optimization of margin [4]. , - . , . - - , .
X , Y ,y : X Y , X = (xi, yi)i=1 X Y ,yi = y
(xi). , - a : X Y , y X.
X rj : XX R+,j = 1, . . . , n, . X n fj : X R, rj(x, x) =
fj(x) fj(x).
,
a(x) = argmaxyY
y(x); y(x) =
iIyBi(x);
y(x) x y; Iy {1, . . . , } - y, ; Bi(x) , - x xi i {1, . . . , n}.
Bi(x) =[i(x, xi) 6 Ri
]; i(x, xi) =
ji
1jrj(x, xi);
j . Bi(x) xi - Ri i(x, xi). , Bi(x) x, Bi(x) = 1. , a(x) Iy, i, xi, Ri, j, y Y , i Iy, j = 1, . . . , n.
-
106 (MM) .., ..
, - xi ( ) - i Ri, Bi(x).
, -. ; - i ; Bi(x) - yi, . . yi ; -, , . . yi , .
. j
j . , -
. ( ). xi [2] i. Ri, Bi(x) - ( ) . , W (Bi), - .
x X
M(x) = y(x)(x) maxyY \{y(x)}
y(x).
M(x), x. -, - , [4].
Bi(x) M(x) - m(x) = Bi(x)
(2 [yi=y
(x)] 1) {1, 0}.
x w(M(x),m(x)), w(M,m) :
( ) M(x) , ;
, , - , , - ;
-
(MM) 107
- , ; , .
- : W (Bi) =
i=1 w(M(xi),m(xi)). -
, , , . -.
Bi(x) U X (Bi, U) = 1|U |
xU
[Bi(x) 6= [y(x) = yi]
].
, Bi(x) X
Xk. - Bi(x) - : (Bi,X,Xk) = (Bi,Xk) (Bi,X).
, . XL , XL = Xn Xkn,L = + k, n = 1, . . . , N . . , , - , : - ; ; ; W (Bi), . - - - . . , , -. , , - . .
UCI - [3]. (promoters) - , , . 1.
-
108 (MM) ..
4.5 4.5 C5.0 RIP- SLIP- Trees Rules Rules PER PER
german 27.5 27.0 28.3 28.6 27.2 25.5australian 18.8 18.8 20.1 15.2 15.7 15.8ionosphere 10.3 10.3 12.3 10.3 7.7 6.7liver 37.5 37.5 31.9 31.3 32.3 32.3promoters 22.7 18.1 22.7 18.1 18.9 5.5breast-cancer 6.6 5.2 5.0 3.7 4.2 3.6hepatitis 20.8 20.0 21.8 19.7 17.4 16.6
1. 10- - 7 UCI.
, 05-01-00877, - .
[1] . . - // . . 1978. . 33. . 568.
[2] . . . : , 1981.
[3] Cohen W. W., Singer Y. A simple, fast and effective rule learner // Proc. ofthe 16 National Conference on Artificial Intelligence. 1999. Pp. 335342.
[4] Mason L., Bartlett P., Baxter J. Direct optimization of margins improves gene-ralization in combined classifiers: Tech. rep.: Australian National Univ., 1998.
..
, ...
- . - (, ) . - , , , .
. - , ( ) - . - N R.
-
(MM) 109
(i, j) - : rij 7 rij . . - A : R 7 R, R , rij = rij ,
1) , . . R R;2) -
;3) R, -
R.
. , 2, . .
( - ) - :
Qw(R, R) = wuQu(R, R) + wpQp(R, R), wu, wp > 0; (1)
Qu(R, R) =
(kl)EN
wkl (rkl rkl)2
(kl)EN
r2kl, wkl > 0; (2)
Qp(R, R) =
(klm)wkl,km
(rklrkm
rklrkm
)2+
+ wkl,lm
(rklrlm
rklrlm
)2+ wkm,lm
(rkmrlm
rkmrlm
)2, (3)
wkl,km, wkl,lm, wkm,lm > 0.
( (2), (3)) , . , .
-. . - . -, ,
-
110 (MM) ..
. - (ijk), (ikl) (jkl), (klm), k, l,m 6 {i, j}. - , . - O(N3). N .
- . , - . , - . -, O(N2), , . . . - .
A. - -A, . . , - , .
R :rij 7 rij rij . , ( ). , - R, A.
1- : (ijk), . -, .
2- : (ikl) (jkl), - . :
rkl = rminkl + (1 )rmaxkl , (k, l) : k, l 6 {i, j},
rminkl = max{|rik ril|, |rjk rjl|}, rmaxkl = min{(rik + ril), (rjk + rjl)}, [0, 1] (ikl), (jkl).
3- : . A
N 2 , (N 2)(N 3) .
-
(MM) 111
O(N), - AO(N2).
, rkl, rkl.
A - , ,, , , . - - . - .
[1] .. - // . 2006. . 46, 2. . 344361.
[2] Deza M., Dutour M. Data mining for cones of metrics, quasi-metrics, semi-metrics, and super-metrics. 2006.
..
,
- . - , - [1, 2]. [3] . - , - .
X , Y = {1, 1} , y : X Y -, X = (xi)i=1 X Y , yi = y(xi). - a : X Y , y(x) X.
, X - W = (wi)i=1
-
112 (MM) ..
b : X R Q(b) =
i=1 wi
[a(xi) 6= yi
], a(x) = sign b(x) -
, b.
b1, . . . , bT - a(x) = F
(b1(x), . . . , bT (x)
), F : RT Y
[1]. F RT Y ,
F (b1, . . . , bT ) - b1, . . . , bT , , , . - ( - ). , , , .
xj , xk X - b1, . . . , bT , yj < yk bt(xj) > bt(xk) t = 1, . . . , T . - [4]. b1, . . . , bT - . bt wi xi, i = 1, . . . , , - b1, . . . , bt1, . , , : = 0 bt - b1, . . . , bt1; = 1 bt , - F (b1, . . . , bt1); (0, 1) .
wi - [4], - . .
- UCI. - : - - -; (SVM). , , AdaBoost [5].
-
(MM) 113
ionospere house-vote bupa diabetes
Monotone (SVM) 9.7% (3) 3.2% (5) 31.3% (2) 23.6% (2)Monotone (Parzen) 8.0% (2) 5.6% (5) 32.7% (3) 30.2% (2)AdaBoost (SVM) 11.5% (65) 4.1% (40) 30.7% (15) 22.7% (15)AdaBoost (Parzen) 12.0% (15) 6.0% (11) 33.0% (34) 29.0% (23)SVM 13.1% 4.5% 42.2% 23.0%Parzen 15.0% 6.2% 33.8% 30.7%
1. , 50. .
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
0.105
0.110
0.115
0.120
0.125
0.130
0.135
0.140
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
11 1 1
11 1
1
1
1
1 11
1 1 11
11 1
2
2
22 2
2 2 2 2
2
2 22
2 2 22
2 2 2
3
3
3 3
3 3 3 3
3 3
3
3 3 3 3 3
3 3 3 3
5
5
5 5
5
5 5
5 5 5
5 5
5 5
5 5 5
5 5
5
6
6
6
6
6
6 66 6
6 66
6
66 6 6
6 6
6
. 1. - SVM ionosphere. 16 T . T = 4, . , - 0, SVM.
: 50 - (80%) (20%). T - . 1.
- 2 3, , -. T .
0.10.4, 1. ( = 1) .
-
114 (MM) ..
T = 2 ; T > 3 . , .
, , , - , - .
, 05-01-00877, - .
[1] . . - // . . 1978. . 33. . 568.
[2] Kuncheva L. Combining pattern classifiers. John Wiley & Sons, Inc., 2004.
[3] . ., . . // .. 1999. . 367, 3. . 314317.
[4] . . - // .2000. . 40, 1. . 166176.
[5] Freund Y., Schapire R. E. Experiments with a new boosting algorithm //International Conference on Machine Learning. 1996. Pp. 148156.
..
,
- -. , . .
- , i , i = 1, . . . , N , n- ( ), - xi = (xi1, . . . xin), X(N,n). Xj == (x1j , . . . xNj)
, j = 1, . . . n. K - (, , ), .
-
... (MM) 115
- (, K- [1], FOREL [2]) - [3] -: xk = xk. , -. xk xi k.
D(N,N) (xk) . k, - k. k = k - , . . x(k) xk.
k , , i, j ; i, j = 1, . . . , N , - cij = (d2ki + d
2kj d2ij)/2,
cii = d2ki dpq = d(p, q). Ck(N,N), k == 1, . . . , N , k i, i = 1, . . . , N . - [4] Ck = XX, Ck(N1, N1) n < N ,X(N 1, n) 1, . . . k1, k+1, . . . N n k. - [5] i , i = 1, . . . , N .
k k, k = 1, . . . ,K . k Ck(N,N). - ckii = d
2(i, k), i = 1, . . . , N k k i , i = 1, . . . , N :
d2(i, k) =1
Nk
Nk
p=1
d2ip 1
2N2k
Nk
p=1
Nk
q=1
d2pq; p, q k,
Nk k. , K- FOREL.
S(N,N) - sij = s(i, j) > 0 i, j xi = x(i), i = 1, . . . , N , - N . i, j k sij = (d2ki+d2kj d2ij)/2. - sii = d2ki, d
2ij = sii + sjj 2sij ,
.
-
116 (MM) ..
K k. - k i :
s(i, k) =1
Nk
Nk
p=1
sip; p k,
Nk k. , K- FOREL.
- :
2k =1
2N2k
Nk
i=1
Nk
j=1
d2ij =1
Nk
Nk
i=1
sii 1
2N2k
Nk
i=1
Nk
j=1
sij ; i, j k.
sij = sij/siisjj 2k = 1 k.
n R(n, n) K- S(n, n), sij = r2ij sij = |rij |, (nk i k):
I1 =K
k=1
nkk =
Kk=1
nki=1
r2(i, k) I2 =K
k=1
nkk =
Kk=1
nki=1
|r(i, k)|. [6]
J1 =K
k=1
nki=1
r2(i, k) J2 =K
k=1
nki=1
|r(i, k)|, i k, k , k k, - K , - . - : K [7].
k, k k - k i k:
s(i, k) = (1/nk)nkj=1
sij ;
s(i, k) =nkj=1
kj sij = kkj , k = (
k1 , . . .
knk);
s(i, k) =nkj=1
sij ;
k , k S(nk, nk) i k.
-
, (MM) 117
, s(i, k) s(i, k) - . . . , J1 > I1. , K- , , -.
FOREL -, 1-. -, -, FOREL, .
, 05-01-00679 INTAS, 04-77-7347.
[1] Tou J.T., Gonzalez R.C. Pattern recognition principles. London: Addison-Wesley, 1981.
[2] . . . -: . - ., 1999.
[3] .. // , : , 1965. . 3845.
[4] Young G., Householder A. S. Discussion of a set of points in terms of theirmutual distances // Psychometrika. 1938. V. 3. P. 1922.
[5] Torgenson W. S. Theory and methods of scaling. N.Y.: J.Wiley, 1958.
[6] .. - // . 1970. 1. . 123132.
[7] Harman H.H. Modern factor analysis. Chicago: Univ. Chicago Press, 1976.
,
.., ..
, ,
- , . , ( -) ( ). - . , -
-
118 (MM) .., ..
[1, 2, 3], -. , - [4]. , - , -, , - . - , . , - , K , K.
G- . , . S1 S2 K G-, - S, K, : (S, S1) 6 (S1, S2) (S, S2) 6 (S1, S2). , - . K , . , G--. - G-. S , S1 S2 , : (S, S1) 6 (S1, S2) (S, S2) 6 (S1, S2).
, - . , . -, 0.25mK , mK K . - S K (S,K) =
SiK V (i)/V0(i),
- K, V0(i) , Si, V (i) , S Si.
-
(MM) 119
- . - , , - . - .
, 06-01-00492,06-01-08045.
[1] .., .., .. . - . . . : , 2006.
[2] .., .., .., .., .., .., .. - : , - . .: , 1998. 63 .
[3] .., .., .. . - - // - , 1996. . 15, 1. . 81100.
[4] Dokukin A.A., Senko .V. About new pattern recognition method for theuniversal program system Recognition. Proc. of the Int. Conf, I.Tech-2004,Varna (Bulgaria), 1424 June 2004. Pp. 5458.
[5] .., .. , // . -. . -12. , 2005, . 200203.
.., .., ..
,
, - . [2]. - - - .
-
120 (MM) .., .., ..
-, -, - , - [3]. (0, 1)-
0 = 0(x;V ) = argmaxi{1,...,M}
qifi(x).
x Rk; V = Mi=1 V (i) N =M
i=1 Ni,
V (i) ={x(i)j Rk i-
}; qi, i = 1, . . . ,M -
i- ; fi(x) - i- fi(x) x Rk,
fi(x) =1
Nick
Ni
j=1
(x x(i)j
c
),
, c . 1 2 :
{x 1, h(x) = ln f1(x)f2(x) < t,x 2, ,
h(x) h(x) = ln(f1(x)/f2(x)), t = ln (q1/q2) .
[2], DB (v1, . . . , vk), - . - .
n(x) S x. DB :
DB =
S
n(x)n(x)f(x)dx
/
S
f(x)dx,
f(x) x. S -
. x(1) x(2) . , ,
-
(MM) 121
. , , x S. .
h(x) = t. h(x) -, S x :
h(x) = hx1
x1 +h
x2x2 + +
h
xnxn
h
x1x1 +
h
x2x2 + +
h
xnxn.
, [2], , . [1] DB , , - .
. M (M > 2) [2] - . DB
DB =
(i,j)
qiqjDB(i,j). (1)
- (. 1).
- ( . 1 ), , , - . - - , (1). .
- , - .
86.
-
122 (MM) ..
I = 2.33% II = 2. %1 5 III = 2. %2 5
. 1. , - . - (III) , (I) (1) (II).
[1] .., .. - - // . . -2006, : , 2006. . I. . 409417.
[2] Lee C., Landgrebe D.A. Decision Boundary Feature Extraction for Non-Parametric Classification // IEEE Trans. on System, Man and Cybernetics. 1993. Vol. 23, N,2. . 433444.
[3] .. . : , 1992. 232 .
..
,
- () .
- .
-
(MM) 123
[2] - [3], . . (, ) . - [1, 4], , , - .
. . ( Sj , Si) |Si Sj |, : ,
(Si, K(Sj)
), K(Sj) -
Sj . . , , - , , ,. . , - . , - - .
- - , - , .
. - , . - , . , - .
, . , ( , ) . , - -
-
124 (MM) ..
, - .
. - A, . A . - . -, : .
, . - , , -, .
, - . - , -. , n- , -. - .
06-01-08045 ,05-01-00332, 05-07-90333, 06-01-00492, - , -5833.2006.1.
[1] .. // 11- - -11, , 2003. . 6871.
[2] .. (-) II // . 1977. 6. . 2127.
[3] .., .. , - // . 1979. . 19,3.
[4] Dokukin A.A. Optimal method for constructing AEC of maximal height incontext of pattern recognition // Pattern recognition and image analysis. 2005. Vol. 15, No. 1.
-
(MM) 125
.., ..
[email protected], [email protected]
,
- , .
. , , . , - . , , - , .
, , - , . -, .
. , , - . .
( ). -. . , . - , .
, - . , , , - 4.5,
-
126 (MM) ..
. 4.5, 4.5.
, 07-01-00516 06-07-89299 5833.2006.1.
[1] .., .. . : , 1992. . 3374.
..
, . ..
() - -, -. , - , (overfitting) -. (pruning) [5] (grafting) [9] , - - - , .
- . - , . - (random decision forest), [6], , . [4] (randomforests). . [8] (lumberjack algorithm) (linked decision forest), . [7] . [1, 2, 3] -
-
(MM) 127
. - . , , , -, , .
- , , - [1, 2, 3].
[1] .., .. r- - // , . 2002. 2. . 5458.
[2] .. VCD r- // . 2003. 2. .3543.
[3] .. - // -. 2006. 1. . 5561.
[4] Breiman L. Random Forests // Machine Learning. 2001. No 45. Pp. 532.
[5] Esposito F., Malerba D., Semeraro G.A. Comparative Analysis of Methods forPruning Decision Trees // IEEE Transactions on Pattern Analysis and MachineIntelligence. 1997. Vol. 19, No 5. Pp. 476491.
[6] Ho T.K. The Random Subspace Method for Constructing Decision Forests //IEEE Transactions on Pattern Analysis and Machine Intelligence. 1998. Vol. 20, No 8. Pp. 832844.
[7] Rouwhorst S. E., Engelbrecht A. P. Searching the Forest: Using Decision Trees asBuilding Blocks for Evolutionary Search in Classification Databases // Congresson Evolutionary Computation CECOO. 2000. Vol. 1. Pp. 633638.
[8] Uther William T.B., Veloso Manuela M. The Lumberjack Algorithm forLearning Linked Decision Forests // Symp. on Abstraction, Reformulation andApproximation, LNAI. Springer Verlag, 2000. Vol. 1864. Pp. 219230.
[9] Webb G. I. Decision Tree Grafting // 15th Int. Joint Conf. on ArtificialIntelligence, Nagoya, Japan. Morgan Kaufmann, 1997. Pp. 846851.
-
128 (MM) ..
..
, . . -
() - ( ), - ( ) - , . - , , () , - . - .
Ax = b, A Rmn, b Rm, x Rn ,X(A, b) , {x |Ax = b} , A . -, X(A, b) = . , , .
( ) :
Z1[1, 2, 3] :[H h]
infX(A+H,b+h) 6=
;
Z2[1, 3] : H infX(A+H,b) 6=
;
H Rmn , h Rm .,
Z0 : h infX(A,b+h) 6=
,
Z1 TLS(Total Least Norm ).
Z1 Z2 -. , ,
-
(MM) 129
, - [4], - [3, 4], () [1, 4, 5], - () - ( ) [3, 4] [4]. - , () , , [6] () [7, 8].
,
- . - -. - [3, 4]. , - , x = A+b, A+ ( -). , [9], x, T A, b, x:
T :
{x min;AA 6 ; b b 6 ; x X(A, b);
> 0, 0 6 < b, - A b - () . , -, Z1 Z2 x[Hh] X(A+H, b+h) xH X(A+H, b), [H h] Z1, H Z2.
, , , -, -, A -. , , - . , ,
-
130 (MM) ..
. , - , , , , , - Z1, Z2 T , - .
, .
1. T < (bAx )/x, -, x , - T . , Ax = b ,x ,
AA 6 ,
b b 6 , lim
,0x = x,
. . x x.
.
1. , b A. (- , - , - , - Z1, , ).
2. - . ( )
3. , -. ( . , ).
[1] .., .., .. - . : , 1983. 336 .
-
(MM) 131
[2] Van Huffel S. Analysis of the total least squares problem and its use inparameter estimation // PhD thesis, Dept. of Electr. Eng., KattholiekeUniversiteit, Leuven, Belgium, 1987.
[3] .., .. - . : , 2004. 193 .
[4] .., .. ( ) - //, -, : , 2004. . 3563.
[5] .. - // . 2001. . 41,11. . 16971705.
[6] .., .., .. - - // . . 2. 2005. . 12, 2. . 322.
[7] Lemmerling P. Structured total least squares: analysis, algorithms andapplications // Ph.D. thesis, Kattholieke Universiteit, Leuven, Belgium, 1999.
[8] .., .., .. // ., . . . , : , 2003. . 7488.
[9] .. - // 1980. . 20, 6. . 13731383.
. ., .., ..,
..
,
, (). [1], - , .
-
132 (MM) . ., .., .., ..
- - GRAD [2]. , (FRiS-) [3]. FRiS-Cluster [4].
- . LOCATEST [5].
: X r ? , : L, L2 L1? d - Z , , . . d- abs-? t d- ? - 12 . - [1]. - .
, R 6 d, -1. K Z N . Z Oi d, i . k < K - , . . 6 .
, M - , abs- Z, -8. , - ,
1. , i R > 2R(zi), Z , Oi.
i M Oj , F = |R(zi) R(ji)|, j = 1, . . . , kl, -. Z - Z . , -
-
(MM) 133
. 23 .
, , , . - FRiS-Cluster.
, , ? ? - ?. - F (FRiS-). Z a b Ra Rb, Z a F (a) = (Rb Ra)/(Ra +Rb). F +1 1. Z - a, Ra = 0 F (a) = 1, F (b) = 1. Ra = Rb F (a) = F (b) = 0, .
FRiS-Cluster M A -. - , A - R. Oi A - , Rj - Oj F (i) = (R Rj)/(R +Rj). Fs - . - , Fs - . k, - Fs(k).
, - GRAD. - . - (Addition) (Deletion) -. - Fs -. Fs U
-
134 (MM) .., .., .., ..
. F - , .. [6].
. , - : ( -), (GRAD) ( FRiS-Cluster).
, 05-01-00241.
[1] . ., .. - // . 2006. . 38, 5. . 5462.
[2] . ., .. GRAD // VIII . . - , : . , 2006. . 8189.
[3] . . , // ( ).
[4] .. FRiS-Tax // ( ).
[5] www.dvv-2.gorodok.net
[6] .. - // . . 1933. 1. . 164167.
,
.., .., ..,
..
,
- - . , - , , , - .
- . ( )
-
, (MM) 135
. - , [1], - [2]. - . [3].
, -. -: , , , - .
, - {xijt}I,J,Ti,j,t=1. xijt j- i- - t.
, A. - A = {U, D, N}, U D - , N . {xijt}Tt=1 - {uijt}Tt=1 {dijt}Tt=1, uijt {U, N},dijt {D, N}, , .
, . xijt1 , . . . , xijtN U, :
N 6 N Z;xijt1 < xijt2 ;
xijtN1 < xijtN ;
D TUij TUkj
},
. Dikj {dijt}Tt=1 {dkjt}Tt=1
. Uikj