introduction to ilp

8/14/2019 Introduction to ILP

1/32

Introduction to ILP

ILP = Inductive Logic Programming

= machine learning logic programming

= learning with logic

Introduced by Muggleton in 1992


2/32

(Machine) Learning

he proce!! by which relatively permanent change!occur in behavioral potential a! a re!ult o" e#perience$(%nder!on)

Learning i! con!tructing or modi"ying repre!entation!o" what i! being e#perienced$ (Michal!&i)

% computer program i! !aid to learn"rom e#perience

Ewith re!pect to !ome cla!! o" ta!&! Tandper"ormance mea!ureP' i" it! per"ormance at ta!&! inT' a! mea!ured byP' improve! with e#perienceE$(Mitchell)


3/32

Machine Learning echniue!

eci!ion tree learning

*onceptual clu!tering

*a!e+ba!ed learning

,ein"orcement learning

-eural networ&! .enetic algorithm!

and/Inductive Logic Programming


4/32

0hy ILP + tructured data

eed e#ample o" 3a!t+0e!t train! (Michal!&i)

What makes a train to go eastward ?


5/32

0hy ILP 4 tructured data

Mutagenicity o" chemical molecule!

(5ing' riniva!an' Muggleton' ternberg' 1996)

0hat ma&e! a molecule to be mutagenic


6/32

0hy ILP 4 multiple relation!

hi! i! related to !tructured data

rain *ar

t1 c11

t1 c12

t1 c17t1 c16

t2 c21

/ /

*ar Length hape %#e! ,oo" /

c11 !hort rectangle 2 none /

c12 long rectangle 7 none /

c17 !hort rectangle 2 pea&ed /c16 long rectangle 2 none /

c21 !hort rectangle 2 "lat /

/ / / / / /

ha!8car car8propertie!


7/32

0hy ILP 4 multiple relation!

.enealogy e#ample

.iven &nown relation!/father(Old,Young)and mother(Old,Young) male(Somebody)and female(Somebody)

/learn new relation! parent(X,Y) :- father(X,Y).

parent(X,Y) :- mother(X,Y).

brother(X,Y) :-

male(X),father(Z,X),father(Z,Y).

Mo!t ML techniue! can:t u!e more than 1 relation

e$g$ deci!ion tree!' neural networ&!' /


8/32

0hy ILP 4 logical "oundation

Prolog = Programming with Logic

i! u!ed to repre!ent

4;ac&ground &nowledge (o" the domain) facts43#ample! (o" the relation to be learned) facts

4heorie! (a! a re!ult o" learning) rules

upport! 2 "orm! o" logical rea!oning4eduction

4Induction


9/32

Prolog + de"inition!


10/32

Logical rea!oning deduction

rom rule! to "act!/

; >+ 3mother(penelope'victoria)$

mother(penelope'arthur)$

"ather(chri!topher'victoria)$

"ather(chri!topher'arthur)$

parent(?'@) + "ather(?'@)$

parent(?'@) + mother(?'@)$

parent(penelope'victoria)$parent(penelope'arthur)$

parent(chri!topher'victoria)$

parent(chri!topher'arthur)$


11/32

Logical rea!oning induction

rom "act! to rule!/

; 3 >+ mother(penelope'victoria)$

mother(penelope'arthur)$

"ather(chri!topher'victoria)$

"ather(chri!topher'arthur)$

parent(?'@) + "ather(?'@)$

parent(?'@) + mother(?'@)$

parent(penelope'victoria)$parent(penelope'arthur)$

parent(chri!topher'victoria)$

parent(chri!topher'arthur)$


12/32

Induction o" a cla!!i"ier

or *oncept LearningMost studied task in Machine Learning

Given

4bac&ground &nowledge B

4a !et o" training e#ample! E

4a cla!!i"ication c C"or each e#ample e

Find a theory T(or hypothesis) !uch that

; >+ c(e)' "or all e 3


13/32

Induction o" a cla!!i"ier e#ample

3#ample o" 3a!t+0e!t train!

B relation! ha&%arand %arpropertie&

(length, roof, &hape, etc$)

e#$ ha&%ar(t1,%11), &hape(%11,bu%et)

E the train! t1to t1*

C ea&t, +e&t


14/32

0hy ILP + tructured data

eed e#ample o" 3a!t+0e!t train! (Michal!&i)

What makes a train to go eastward ?


15/32


3#ample o" 3a!t+0e!t train!

B relation! ha&%arand %arpropertie&

(length, roof, &hape, etc$)

e#$ ha&%ar(t1,%11)

E the train! t1to t1*

C ea&t, +e&t

Po!!ible Tea&t() :-

ha&%ar(,), length(,&hort), roof(,).


16/32


3#ample o" mutagenicity B relation! atomand bond

e#$atom(mol!",atom1,%,1/).

bond(mol!",atom1,atom",0).

E 27A molecule! with &nown cla!!i"ication

C a%ti'eand nona%ti'ew$r$t$ mutagenicity

Po!!ible Ta%ti'e(ol) :-

atom(ol,2,%,!!), atom(ol,3,%,1*),

bond(ol,2,3,1).

%!!

%1*


17/32

Learning a! !earch

Given

4 ;ac&ground &nowledge ;

4 heory e!cription Language T4 Po!itive! e#ample! P (cla!! B)

4 -egative e#ample! - (cla!! +)

4 % covering relation covers(B,T,e)

Find a theory that cover!

4 all po!itive e#ample! (completene!!)

4 no negative e#ample! (con!i!tency)


18/32

Learning a! !earch

*overing relation in ILP

cover!(;''e); >+ e

% theory i! a !et o" rule! 3ach rule i! !earched !eparately (e""iciency)

% rule mu!t be con!i!tent (cover no

negative!)' but not nece!!ary complete Separate-and-conquer!trategy

4,emove "rom P the e#ample! already covered


19/32

pace e#ploration

trategy

,andom wal&

4 ,edundancy' incompletene!! o" the !earch

y!tematic according to !ome ordering

4 ;etter control =C no redundancy' completene!!

4 he ordering may be u!ed to guide the !earch toward!

better rule!

What kind o ordering?


20/32

.enerality ordering

,ule 1 i! more general than rule 2

=C ,ule 1 cover! more e#ample! than rule 2

4 I" a rule i! con!i!tent (cover! no negative!)then every !peciali!ation o" it i! con!i!tent too

4 I" a rule i! complete (cover! all po!itive!)

then every generali!ation o" it i! complete too

Mean! to prune the !earch !pace 2 &ind! o" move! !peciali!ation and generali!ation

*ommon ILP ordering D+!ub!umption


21/32

.enerality ordering

parent(X,Y):-

parent(X,Y):- female(X) parent(X,Y) :- father(X,Y)

parent(X,Y) :- female(X), mother(X,Y)

parent(X,Y) :- female(X), father(X,Y)

%on&i&tent rule&pe%iali&ation


22/32

earch bia!e!

!"ias reers to any criterion or choosing one genera#i$ation

over another other than strict consistency with the

o%served training instances&' (Mitche##)

,e!trict the !earch !pace (e""iciency)

.uide the !earch (given domain &nowledge)

i""erent &ind! o" bia!

4 Language bia!

4 earch bia!

4 trategy bia!


23/32

*hoice o" predicate!

roof(,flat) 4 roof() 4 flat() 4

ype! o" predicate! ea&t() :- roof(), roof(,")

Mode! o" predicate!

ea&t() :- roof(,flat)ea&t() :- ha&%ar(,), roof(,flat)

i!cretiEation o" numerical value!

Language bia!


24/32

earch bia!

he move! direction in the !earch !pace

op+down

4!tart the empty rule (c(?) + $)4move! !peciali!ation!

;ottom+up

4!tart the bottom clau!e (F c(?) + ;$)4move! generali!ation!

;i+directional


25/32

trategy bia!

Geuri!tic !earch "or a be!t rule

Gill+climbing4 5eep only one rule

4 e""icient but can mi!! global ma#imum

;eam !earch4 al!o &eep krule! "or bac&+trac&ing

4 le!! greedy

;e!t+"ir!t !earch4 &eep all rule!

4 more co!tly but complete !earch


26/32

% generic ILP algorithm

procedure567(Examples)

5nitiali8e(Rules, Examples)

repeat

R9 Sele%t(Rules, Examples)

Rs9 efine(R, Examples)

Rules9 edu%e(Rules;Rs, Examples)untilStoppingriterion(Rules, Examples)

return(Rules)


27/32

% generic ILP algorithm

< 5nitiali8e(Rules,Examples):initialiEe a !et o"theorie! a! the !earch !tarting point!

< Sele%t(Rules,Examples):!elect the mo!t promi!ingcandidate rule R

< efine(R,Examples):return! the neighbour! o" R(u!ing !peciali!ation or generali!ation)

introduction to ilp

Documents