what is shannon information
TRANSCRIPT
-
8/10/2019 What is Shannon Information
1/33
1
A , ( A 2013), 20
. N ,
. A L F (2010, 2011) ,
, , , . I 20 ,
A , . I ,
(B H C 1953, B H 1964, , :
, . O
. A , ,
( ; F 1925), (
C 1987), N (
; S N ,
C S (1948)
-
8/10/2019 What is Shannon Information
2/33
2
W 1949). R , S
.
A , ( A B
, ,
, . I . H , . I
S
M ,
I , S 2 S . I S 3,
S 4
S 5. I S 6, . T
(S 7), T
S , , 9. T , S 10,
, .
W T M T C (1948), S
-
8/10/2019 What is Shannon Information
3/33
3
. S (K
A S (1948; S W 194
: A , A ,
I , A CH , ,
. A , .
A D, .
T , . T
:
= = (1)
S , , :
= =
= = (2)
A , D , . T
: = = (3)
-
8/10/2019 What is Shannon Information
4/33
4
A D :
= =
= = (4)
I , S (1948, . 349) :
;
;
, . W
I . N ,
. H , J N
: , . I . I
, ,
. (T M I 1971, . 180). I I : , , , , . I ,
,
T :
H ( ) H ( D)
H ( ; D) E
-
8/10/2019 What is Shannon Information
5/33
5
:
: D.
: D. : D .
A ,
= = (5)
E E D:
I D , E ( =
= ), ( = ).
I D , E ( = = ), ( = = ).
T E CH . T : CH
, D ,
, E :
= =
= (6)
= =
= (7)
= . T C :
= (8)
-
8/10/2019 What is Shannon Information
6/33
-
8/10/2019 What is Shannon Information
7/33
7
M , ,
. ., B S 1997, F 2013). M , ,
(M K 1969; N 1972; D 1981). A , S , ,
. S , ,
S S
. B :
?D ,
)
T (2004, 2008, 2013) A A
, ,
. . (T 2004, . 3). H : , ,
. (T 2004, . 22; T 2008, 2013). T , ,
(T 2008, . 25). O , : ,
, , , ( T 2004, . 21 22; 2
-
8/10/2019 What is Shannon Information
8/33
8
A , : , ,
? T
: , ,
; 17 .
.
, , , .(2004, . 21 22). B : S ,
.(S 1948, . 379; ). T
17, 17 . A , D
, :
. I , 17 D ,
. T ,
I , A D
. (D 2008, . 199). T ;
( . . 199), ( ) S
( . . 202), ( )
S ( . 203). B , S ,
, ( . . 200).
-
8/10/2019 What is Shannon Information
9/33
9
T , : ,
E . I
. T :
( W 2011).
W T , T (2004, . 20 21), ,
(D 2008, . 201), ,
( ., . 201). H , S ,
.
S , , S
A :
. O , S
I S 2,
,
,
( )
-
8/10/2019 What is Shannon Information
10/33
10
( . (1) (3)). H , S ,
. (2) (4), .
O , ( = ). F , T : I
, . (T 2004, . 11).T S ;
S 6.
A S F D
, S ,
: ,
. F
. (D 1981, . 48). F ,
,
( ., . 52).
D , , S ,
, ,
.
(D 1981, . 52). H , S ,
, D (T 2004, L 2005),
-
8/10/2019 What is Shannon Information
11/33
-
8/10/2019 What is Shannon Information
12/33
12
F
, . H , ,
:
. N ,
S . T ,
. S : I . F ,
. ,
. (D 2003, . 486). B :
( )
. (T 2006, . 592).
A , ,
. F , P M ,
B I S , P . A S (
) L (
I , C C C F (1996),
S , ,
). T ,
-
8/10/2019 What is Shannon Information
13/33
13
, , ,
.
A , ,
,
I ,
. L .
T ,
{ } = , ; (. T ,
2 . I , { } = ,
, { } = , ; . T
: = . I , (
). F ,
; = , .
I S , , . I ,
: , ,
-
8/10/2019 What is Shannon Information
14/33
14
. T . T , L :
=
= (9)
L : L , . T N C
, , L L:
= (10)
, , 2.
T
, , ,
, O
, =
), = =
,
L ( . (9)).
I ,
. F , ( ), T S : (
1948) ( , 1995) .
, . (T 2008, . 23; ). F , F
S :
-
8/10/2019 What is Shannon Information
15/33
15
? (T 2008, . 27; ), ,
. (T 2006, . 592).A , S S :
. (T 2004, . 19).
T S , . F ,
. S ,
F , , S? I ,
: ? I , , . (3), , ,
(3) : ,
M , ( , ),
?,
? F , ,
; ,
. I ,
?
T S . A S 9
-
8/10/2019 What is Shannon Information
16/33
16
, ,
.
A S 3, S . H ,
A . I ,
.
T . F ,
: (B 1957, .
7). B ,
. F , 37 ,
: , . T
.
B S 3, S
: , D
,
. (D 2008, . 201).
-
8/10/2019 What is Shannon Information
17/33
17
I S . P
: S
. T
, . T
, , . I
.
F 1948 ,
S . H , , . A
B S (1995) S N S S . S
, S
A B S , ,
. B (S 1948, . 393; B, T 2003),
:
-
8/10/2019 What is Shannon Information
18/33
18
,
0 1 ,
.(B 2001, . 1; ). B , T
, S . I ,
, . B
: , ; ,
,
.F ,
S ,
. A , :
; (
S ,
, , ,
. (D 2003, . 480).
O S , . I ,
, S
E ( . (7)), ( D, . (5))
-
8/10/2019 What is Shannon Information
19/33
-
8/10/2019 What is Shannon Information
20/33
20
: , ,
, A . T
, , . S ,
.
T ,
S ( T 2004, 2013), : . F , S
D : , , . (1981, . 47). S
, , . I , J M. D
, (2001, . 423). A . F
, . ., , (1999, . 633) , B ,
, ,
. (2009, . 681 682). I S
(B
1957, . 7). A S F B ,
( C , F S 2
A
-
8/10/2019 What is Shannon Information
21/33
21
: , ,
. C
(D 1981, . 77).I , , ,
, W
,
, , ,
( , , . A
,
,
. ., . (5)),
A . T ( ,. ., R 1996)
;
. F , , ,
I ,
:
, ,
-
8/10/2019 What is Shannon Information
22/33
22
; . I
, , , , ,
. (1996, . 188; L 1991). T ; , P K
. (1989, . 37). T
, ;
(S 1990, 1996).
A , T ,
. N , S
. H , . I
, , ,
. A , ,
; , . M ,
T ,
. F , (S 1982, B 1987, K 198
. F ,
-
8/10/2019 What is Shannon Information
23/33
23
. A , :
( L 2004).
L . I ,
, . N , A
,
. H
, . I ,
: . . .
. I ,
,
(D 1981, . 38). T , . N ,
: , .
A , . I ,
, , , ,
. I , ; ,
D
-
8/10/2019 What is Shannon Information
24/33
-
8/10/2019 What is Shannon Information
25/33
25
,
: (F (R 1948, D 1992),
1998). P ,
(P 1991, M P 1993 , C E , C
E ( , W 2013). T
. I ,
. H S ,
. (S 1948, . 379). T
; , , ,
.
I , S ,
. H , , . T , ;
. T
-
8/10/2019 What is Shannon Information
26/33
26
A K (1957) F R (1961), . B
T C J T E I (1991),
I :
... .... F
. I ,
( ), (K
), ( :
( (C
T 1991, . 1).F ,
A B, = = , , = = , , (
). O , , . O ,
, , , . I ,
. S ,
, , , ,
.
O ,
. B
. A , . T ,
-
8/10/2019 What is Shannon Information
27/33
27
.
T
: . C
,
I , . T
: ,
, .
F ,
, , ,
:
( . . H , A
(M K 1961981)
I , , ,
, . B
: , B .
F , ,
-
8/10/2019 What is Shannon Information
28/33
28
. T , ( , . ., C T 1991, C 6) ( , . ., C T 1991,
S , ,
. E
, S:
. ... I
. (S 1993, 180).
D , S
. M ,
, B S (1995)
. F , S ,
, .
A , P. (2013). I . I E. N. ( .), E (F 2013 E ), URL < :// . . / / 2013/ / />
-
8/10/2019 What is Shannon Information
29/33
29
A , P. B , J. ( ) (2008). H I .A : E S P .
B H , Y. (1964). L I : E
A . R , MA: A W .B H , Y. C , R. (1953). S I . B J
, : 147 157.
B , J. S , J. (1997). I F . L D .C C U P .
B , D. (1957). I E A . L : P & S .
B , H. (1987). . O : O U P .
B , . , A. (2001). C I SQ M . A, : #022113.
B , . , A. (2009). I I Q F , : 677 689.
C , H. N. (1984). C , C , E . P . 17 27, P. F H. W ( .), I . M : U
M P .
C , C. M. F , C. A. (1996). Q I : H M IV ? I A. M M. R ( .),D E ,
60 L . A I , : 226 257 ( /9601025).
C , C. M., F , C. A., S , R. (2002 ). U Q S : F R . J , : 4537 4559.
C , G. (1987). A I . N Y : C U PC , T. T , J. A. (1991). E I . N Y : J W &
S .
D , D. H , P. (2000). I F E Q L A, : 1759 1774.
-
8/10/2019 What is Shannon Information
30/33
30
D , P. (1992). W S ' P T C CT . , : 195 216.
D , F. (1981). K & F I . C MA: MIT P .
D , J. M. (2001). T C I D M423 427, W. S ( .), A
L . B : G .
D , A. (2003). Q I D N E . H , : 479 499.
D , A. (2008). Q I D E . H , : 195 216.
E , D. (1986). T T T C ., : 249 58
F , D. (1979). C F E . E , : 219 250.
F , R. (1925). T S E . C , : 700 725.
F , L. (2010). I A I . O : O U P
F , L. (2011). I , O ; O U P .
F , L. (2013). S C I . I E. N. E (S 2013 E ), URL < :// . . / / 2013/ /
H , R. (1928). T I . B J , : 535 563.
H , E., A , L. T , G. (2013). Q C E S C B M . A , : 19790 19795.
J , M. (1974). . N Y : J W & S .
J , R. (1998). E Q C . P . 369 79, M , K. P. T , S. T. T , N. M. J. W ( .), G .O : O U P .
J , R. (2004). I C Q I . IB J D , : 79 85.
-
8/10/2019 What is Shannon Information
31/33
-
8/10/2019 What is Shannon Information
32/33
-
8/10/2019 What is Shannon Information
33/33
, A. (1999). A F P Q M F , : 631 643.