knowledge representation

Knowledge RepresentationÉcole Supérieure Polytechnique

Dakar, SenegalWinter 2011

by Fabian M. Suchanek

This document is available under aCreative Commons Attribution Non-Commercial License

2

Organisation• 1h class on Knowledge Representation

• Slides are available at the Web-site http://suchanek.name/work/teaching

• Language of the Slides is English, lecturer will do his best to talk in French, questions are welcome throughout the session in French or

English

3

CheckTo help assess your background knowledge, please answer the following questions on a sheet of paper:

1. say “All men are mortal” in first order logic.2. reduce ~ x: y: (loves(x,y) & married(x,y))3. write a rule in PROLOG.4. what is the minimum of f(x)=x2 ?

Do not worry if you cannot answer. Hand in your answers anonymously. This check just serves to adjust the level of the lecture. You will not be graded.

E A

4

Definition of KR

Knowledge representation (KR) is a research field of artificial intelligence that aims to formalize information so that it can be used for automated reasoning.

5

MotivationIs Elvis

Presley still alive?

alive(Elvis) ?

sings(Elvis)bornIn(Elvis, Tupelo)locatedIn(Tupelo, USA)

sings(X) => alive(X) …

Elvis is alive / not alive

formalization

reasoning

KR is required to find formal answers to questions:

6

Big PictureIs Elvis

Presley still alive?

Knowledge Base (KB)

facts / assertions

query

rules / formulas

implied / deduced facts / statements

inference

alive(Elvis)

sings(Elvis)bornIn(Elvis, Tupelo)locatedIn(Tupelo, USA)

sings(X) => alive(X) …

7

Query AnsweringIs Elvis

Presley still alive?

Knowledge Base (KB)quer

y

alive(Elvis)

Answering a query:Finding out whether the query or its contrary can be deduced from the KB

sings(Elvis)bornIn(Elvis, Tupelo)locatedIn(Tupelo, USA)

sings(X) => alive(X) …

8

KR FormalismKnowledge Base (KB)

KR formalism: The language of the KB

sings(Elvis)bornIn(Elvis, Tupelo)locatedIn(Tupelo, USA)

sings(X) => alive(X) …

9

KR Formalisms, 1/2There are different ways to represent knowledge:

• PROLOG-like: alive(Elvis), is(Elvis, alive)

• graphically

• in natural language“Elvis is alive”

isalive

• in propositional logic elvis_alive .

10

KR Formalisms, 2/2

• in a programming language elvis.alive=true

• in first order logic x: rocksinger(x) => alive(x)A

• in a mathematical notation elvis Alive

• in completely different formalism ф elvis → ☺☺☺

11

CanonicityA KR formalism is canonic if one piece of knowledge can be represented in only one way

alive(Elvis)is(Elvis, alive)alive(Elvis, true)vivant(Elvis)

not very canonic

12

Canonicity is DesirableCanonicity is in general desirable because it facilitates co-operationby different people

alive(Elvis) ?

is(Elvis, alive)

is(Elvis, alive) !

no

13

Achieving CanonicityA formalism can be made more canonic by• restricting it

e.g., by allowing only unary predicates:alive(Elvis)is(Elvis, alive)

• providing best practice guidelinese.g., by prescribing certain conventions:

alive(Elvis)Alive(elvis)

• providing standard vocabulariese.g., by listing predicates that should be used in a certain domain:{alive, dead, young, old, happy, sad}

14

ExpressivenessA KR formalism is more expressive than another one if we can say things in the first formalism that we cannot say in the second.

First order logic

x: rocksinger(x) => alive(x)A

Propositional logic

?

15

Expressiveness is DesirableIn general, a higher expressiveness is desirable from a modeling point of view

x: alive(x)A

… but it comes at a cost…

16

DecidabilityA KR formalism is decidable, if there is an algorithm (computer program) that can answer any query on a knowledge base in that formalism.

Some formalisms are so expressive that they are undecidable:

Is sentence (*) true?

(Technically: A logical system is decidable iff there is an effective method for determining whether arbitrary formulas are theorems of the logical system.)

(*) This sentence is false.

• Natural language is undecidable

17

Decidability of FOL• First order logic is also undecidable

First order logic is so powerful that it can express sentencesof which it is impossible to determine whether they are true or not.

This means that there are some first-order-logicsentences, which we can never prove or disprove.

Worse, we cannot even determine whethera given sentence is of this type.

18

Achieving DecidabilityIn general, decidability is desirable. But the more expressive a formalism is, the more likely it is to be undecidable. Some formalisms can be made decidable by restricting them• Propositional logic is decidable• First order logic is decidable if all formulae are of the following form:

x, y,…. z, q,… : p(x,y) … => … AE

existential universal arbitrary formulaquantifiers quantifiers without quantifiers and function symbols

19

Closed World AssumptionA KR formalism follows the closed world assumption (CWA), if any statement that cannot be proven is assumed to be false.

PROLOG, e.g., follows the CWA:?- assert(bornIn(Elvis, Tupelo)). yes?- alive(Elvis). no

20

Open World AssumptionIn many contexts, the open world assumption (OWA) is more appropriate.

Under the OWA, a statement can be• provably false• provable true• unknown

?- alive(Elvis). I have no clue

21

ReificationA KR formalism allows reification,if it can treat statements like entities.

thinks(Fabian, alive(Elvis)).

=> alive(Elvis)

reification, a statement appears as argument

22

Modal LogicModal logic allows talking about necessity and possibility

“It is possible that Elvis is alive”

alive(Elvis)

Thus, modal logic implements a (restricted) type of reification.

23

Unique Name AssumptionA KR formalism follows the unique name assumption (UNA), if different names always refer to different objects.

ElvisThe King The King

24

Unique Name AssumptionThe UNA is not useful if different people want to use different identifiers:

alive(TheKing)TheKing=Elvis

=> alive(Elvis)

Elvis The King

25

SchemaA KR formalism is schema-bound, if one has to decide upfront which entities can have which properties.

In schema-bound formalisms, one has to decide a priori for classes of things and their properties:

person:• alive/dead• birthday• profession

camera:• resolution• shutter speed• weight• gender

26

Advantages of Schemas

?- assert(alive(Elvis)). yes?- assert(hasResolution(Elvis,five_megapixel)) yes

PROLOG is schema-free, any entity can have any property:

A schema-bound formalism puts more modeling constraints, but can exclude non-sensible statements.

27

DatabasesDatabases are a particular schema-bound KR formalism.A database can be seen as a set of tables.

Name Profession

Birth

Elvis Singer 1935

Obama President 1961

… … …

each table corresponds to one class of things

Person

each column is one property

each row is one thing

28

SQLA database can be queried in the Structured Query Language (SQL).

SELECT name, birth FROM personWHERE profession=‘Singer’AND birth>1930

Elvis, 1935JohnLennon, 1940…

Name Profession

Birth

Elvis Singer 1935

Obama President 1961

… … …

29

Uses of DatabasesDatabases are used in practically every major enterprise, with the main database systems being• Oracle• Microsoft SQL Server• IMB’s DB2• Postgres• MySQL

Headquarters of Oracle inRedwood Shores, CA, USA

30

InheritanceA KR formalism supports inheritance, if properties specified for one class of things can be automatically transferred to a more specific class.

person:• name• birthday• gender

singer:• name• birthday• gender• instrument

31

Inheritance

person:• name• birthday• gender

singer:

inherits from person • instrument

more general class,fewer properties

more specific class,more properties

additional properties

32

Restrictions

person:• name• birthday• gender• profession

singer:

inherits from person • profession=singer• instrument

restriction

The more specific class can restrict properties inheritedfrom the more general class.

33

Object Oriented LanguagesObject-oriented programming languages (such as e.g., Java)support inheritance.

public class Person { String name; String profession;}

public class Singer extends Person { String profession=“Singer”; String instrument;}

Singer elvis=new Singer();elvis.name=“Elvis”;elvis.instrument=“guitar”;System.out.println(elvis.profession) Singer

super-class* declares two properties

sub-class* overwrites a property* adds a property

Elvis is a singer* but he inherited properties* and has predefined values

34

Object Oriented LanguagesMost programming languages support object-orientation today, with the most important ones being• Java• C++• C#• Visual Basic• Python• Ruby

35

MonotonicityA KR formalism is monotonous, if adding new knowledgedoes not undo deduced facts.

elvis_is_personelvis_aliveelvis_dead => ~elvis_alive

+

Monotonicity can be very counter-intuitive.It requires everything to be known upfront.

First order logic and propositional logic are monotonous:

=> elvis_alive

elvis_is_personelvis_aliveelvis_dead=> ~elvis_aliveelvis_is_dead=> elvis_alive

=> m_jackson_alive=> elvis_is_dead

36

Default LogicDefault logic is not monotonous:

elvis_is_person: elvis_is_alive elvis_is_alive

=> elvis_is_alive

elvis_is_dead~elvis_is_alive

if Elvis is a personand nothing says he’s not alive then he is alive

if Elvis is deadthen he is not alive

elvis_is_person

elvis_is_dead+

prerequisiteconclusionjustification

37

FuzzinessA KR formalism is fuzzy, if certain statements can hold to a certain degree.

fantastic(Bush) 0.1

fantastic(Madonna) 0.8

fantastic(Elvis) 1.0

The opposite of fuzzy is crisp.First order logic, PROLOG and propositional logic are all crisp.

38

Fuzzy LogicFuzzy logic is a fuzzy KR formalism.

rainy => bad_weather

rainy (0.8)

bad_weather (0.8)

39

Fuzzy Logic Operators

rainy \/ windy => bad_weather

rainy (0.8)

bad_weather (??)

windy(1.0)

Fuzzy logic defines how to compute fuzzy values for complex combinations of fuzzy predicates, e.g.• OR is computed as maximum• AND is computed as minimum• NOT is computed as 1-x

1.0

40

ContradictionsA KR formalism is tolerant to contradictions if it allows contradictions in the knowledge base.

Elvis is alive

Elvis is not alive

Knowledge base

Elvis is a person

Madonna is alive

41

FOL & Contradictions

alive(Elvis).~alive(Elvis).

=> life_is_beautiful

First order logic and propositional logic are not tolerant to contradictions:

ex falso quod libet…

Some domains require handling of contradictions(e.g. information extraction from the Web)

Elvisfans.com:Elvis is alive

Wikipedia:Elvis is dead.

42

Possible WorldsGiven propositional predicates, a possible world is an assignment of truth values to the predicates.

w1:elvis_alive elvis_dead

w2:elvis_alive elvis_dead

w4:elvis_alive elvis_dead

w3:elvis_alive elvis_dead

Given the predicates elvis_alive, elvis_deadthere are 4 possible worlds:

43

Markov Logic

f1: elvis_alive [0.8]f2: elvis_dead => ~elvis_alive [0.5]f3: elvis_dead [0.6]

A Markov Logic Network is a set of propositional logic formulas with weights:

A Markov Logic Network assigns a probability to each possible world: P(w) ~ Π eweight(f)

satisfied formula f

w1:elvis_alive elvis_dead

f1 satisfied, f2 satisfied, f3 not satisfiedP(w1) ~ e0.8 . e0.5 = 3.6

44

Markov Logic Example

f1: elvis_alive [0.8]f2: elvis_dead => ~elvis_alive [0.5]f3: elvis_dead [0.6]

w1:elvis_alive elvis_dead

w2:elvis_alive elvis_dead

w4:elvis_alive elvis_dead

w3:elvis_alive elvis_dead

f1,f2 f2 f2,f3

f1,f3

Satisfied formulas:

Markov Logic Network:

Possible worlds:

3.6 1.6 3.0 4.1

45

Markov Logic SummaryA Markov Logic Network is a KR formalism that allowscontradictions.

It assigns a probability to all possible worlds:• larger probabilities for worlds that satisfy more constraints• smaller values for worlds that violate more constraints

46

ExplicitnessA KR formalism is explicit, if it can explain why a certain statement was deduced.

rich(BillGates)

rich(X) => happy(X)

rich(BillGates) => happy(BillGates)

happy(BillGates)

Instantiation

Mod

us

Pone

ns

Logic-based formalisms are usually explicit:

47

ExplicitnessExplicitness may not always be useful, consider pattern recognition:

a a a a a

hard to describe explicitly

a

... or image recognition:

hard to describe explicitly

Elvis Presley

48

Neural NetworksA neural network is a a non-explicit KR formalism.It tries to model the human brain.

connections

neuron(nerve cell)

49

PerceptronsA perceptron is a simplified model of a nerve cell (neuron).It is defined by a vector of n real-valued weights, w1,...,wn.It can be visualized as follows:

nerve cell

x1 x2 ... xn

input values

perceptron

y real-valued output value

w1 w2... wn

weights

50

Perceptron FunctionA perceptron computes a function Rn {0,1}from the n input values to the set {0,1} as follows:

1 if wi · xi > 00 else

x1 x2 ... xn

y =

w1 w2... wn

51

Perceptron Example 1

x1 x2 x30.3 0.8 0.2

wi · xi = 0.3· 1 + 0.8· 2 + 0.2· 1 = 2.1> 0~~~~> Output y=1

1 if wi · xi > 00 else

y =

input: 1 2 1

52

Perceptron Example 2

x1 x2 x30.3 0.8 0.2

wi · xi = 0.3· 0 + 0.8· -2 + 0.2· 0 = -0.8< 0~~~~> Output y=0

1 if wi · xi > 00 else

y =

input: 0 -2 0

53

Combining Perceptrons

final output of the neural network1

Perceptrons can be combined to a neural network.

input of the neural network

output of one perceptron is the input for another one

54

Imitating the Retinaimitation of our eye retina

1

55

Network Traininga1

net will also recognize this one

a

Given examples:a a a a a...adjust the weights so that

the output is 1 for all examples(= training the net).

Then the net will also recognizesimilar inputs:

56

Neural Networksa1

A Neural Network is a non-explicit KR model.

Neural Networks are used, e.g,• to recognize postal codes on letters• to optimize search engine results

This network “knows” what the letter “a” is.

But this knowledge is not explicit.

57

DistributednessA KR formalism is distributed, if it encourages use and co-operation by different people and systems across different places and organizations.

Every rock singer is alive!

The King is a rock singer.

I am Elvis, the King.

Elvis is alive

58

The Semantic WebThe Semantic Web is a distributed KR formalism.

1935

GrammyAward

sameAs

born

won

59

KR Design: SummaryIn general, a KR formalism serves to• represent knowledge• infer facts from that knowledge• answer queries on the knowledge

60

KR Design: SummaryThere are many KR formalisms with different properties:• Canonicity (does the formalism allow only one form to represent a statement?)• Expressiveness (how powerful is the formalism)• Decidability (can every query be answered?)• Closed World (does the formalism assume that everything unknown is false?)• Unique Name (do two names always mean two different things?)

61

KR Design: Summary• Schema-bound (do we have to decide upfront on properties of things?) Databases/SQL is a schema-bound KR formalism• Inheritance (can properties be transferred from one class to another?) Object-oriented programming languages support inheritance• Monotonicity (will new knowledge never undo existing knowledge?) Default logic allows non-monotonicity

62

KR Design: Summary• Fuzziness (can properties be fufilled to a certain degree?) Fuzzy logic is a fuzzy KR formalism• Tolerance to contradictions (is it OK to have a contradiction in the KB?) Markov Logic Networks can deal with contradictions• Explicitness (is knowledge always explainable?) Neural Networks store knowledge only implicitly• Distributedness (can the knowledge be distributed over sources?) The Semantic Web is a distributed formalism