explainable, data-efficient, verifiable representation ...kc2813/xaiseminar/... · barack obama is...
TRANSCRIPT
![Page 1: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/1.jpg)
Explainable, Data-Efficient, Verifiable Representation Learning
Pasquale Minervini, UCL [email protected]
ICL, Nov 27th
![Page 2: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/2.jpg)
Symbolic and Sub-Symbolic AI
![Page 3: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/3.jpg)
Symbolic
Ontologies, First-Order Logic, Logic Programming, Knowledge Bases, Theorem Proving..
∀X, Y, Z : grandFather(X, Y ) ⇐ father(X, Z), parent(Z, Y )
Symbolic and Sub-Symbolic AI
![Page 4: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/4.jpg)
Symbolic
Ontologies, First-Order Logic, Logic Programming, Knowledge Bases, Theorem Proving..
∀X, Y, Z : grandFather(X, Y ) ⇐ father(X, Z), parent(Z, Y )
Symbolic and Sub-Symbolic AI
✓Data-Efficient
✓Interpretable and Explainable
✓Easy to Incorporate Knowledge
✓Verifiable
![Page 5: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/5.jpg)
Symbolic
Ontologies, First-Order Logic, Logic Programming, Knowledge Bases, Theorem Proving..
∀X, Y, Z : grandFather(X, Y ) ⇐ father(X, Z), parent(Z, Y )
Sub-Symbolic/Connectionist
Neural Representation Learning, (Deep) Latent Variable Models..
f1( f2(…fn( )…)) = {0.15 ≡ cat0.85 ≡ dog
Symbolic and Sub-Symbolic AI
✓Data-Efficient
✓Interpretable and Explainable
✓Easy to Incorporate Knowledge
✓Verifiable
![Page 6: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/6.jpg)
Sub-Symbolic/Connectionist
Neural Representation Learning, (Deep) Latent Variable Models..
f1( f2(…fn( )…)) = {0.15 ≡ cat0.85 ≡ dog
✓Noisy, Ambiguous, Sensory Data
✓Highly Parallel
✓High Predictive Accuracy
Symbolic
Ontologies, First-Order Logic, Logic Programming, Knowledge Bases, Theorem Proving..
✓Data-Efficient
✓Interpretable and Explainable
✓Easy to Incorporate Knowledge
✓Verifiable
∀X, Y, Z : grandFather(X, Y ) ⇐ father(X, Z), parent(Z, Y )
Symbolic and Sub-Symbolic AI
![Page 7: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/7.jpg)
Sub-Symbolic/Connectionist
Neural Representation Learning, (Deep) Latent Variable Models..
f1( f2(…fn( )…)) = {0.15 ≡ cat0.85 ≡ dog
✓Noisy, Ambiguous, Sensory Data
✓Highly Parallel
✓High Predictive Accuracy
Symbolic
Ontologies, First-Order Logic, Logic Programming, Knowledge Bases, Theorem Proving..
✓Data-Efficient
✓Interpretable and Explainable
✓Easy to Incorporate Knowledge
✓Verifiable
∀X, Y, Z : grandFather(X, Y ) ⇐ father(X, Z), parent(Z, Y )
Symbolic and Sub-Symbolic AI
![Page 8: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/8.jpg)
Knowledge Graph — graph structured Knowledge Base, where knowledge is encoded by relationships between entities.
Knowledge Graphs
![Page 9: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/9.jpg)
Knowledge Graph — graph structured Knowledge Base, where knowledge is encoded by relationships between entities. In practice — set of subject-predicate-object triples, denoting a relationship of type predicate between subject and object.
subject predicate object
Barack Obama was born in Honolulu
Hawaii has capital Honolulu
Barack Obama is politician of United States
Hawaii is located in United States
Barack Obama is married to Michelle Obama
Michelle Obama is a Lawyer
Michelle Obama lives in United States
Knowledge Graphs
![Page 10: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/10.jpg)
Knowledge Graphs
![Page 11: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/11.jpg)
Knowledge Graphs
![Page 12: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/12.jpg)
Link Prediction in Knowledge Graphs
Washington
Malia Ann Obama
Sasha Obama
Barack Obama
Michelle Obama
lives in
parent of
?
![Page 13: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/13.jpg)
Rule-Based Link Prediction
∀X, Y, Z :married with(X, Y) ⇐
parent of(X, Z),parent of(Y, Z)
Washington
Malia Ann Obama
Sasha Obama
Barack Obama
Michelle Obama
lives in
parent of
?
![Page 14: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/14.jpg)
✕ Not always true ✕ Hard to learn from data ✕ Hard to formalise for other modalities
Washington
Malia Ann Obama
Sasha Obama
Barack Obama
Michelle Obama
lives in
parent of
?
∀X, Y, Z :married with(X, Y) ⇐
parent of(X, Z),parent of(Y, Z)
Rule-Based Link Prediction
![Page 15: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/15.jpg)
Washington
Malia Ann Obama
Sasha Obama
Barack Obama
Michelle Obama
lives in
parent of
?
Neural Link Prediction
![Page 16: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/16.jpg)
P(BO married MO) ∝
fmarried( , )
Washington
Malia Ann Obama
Sasha Obama
Barack Obama
Michelle Obama
lives in
parent of
?
Neural Link Prediction
![Page 17: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/17.jpg)
Learning Representationsℒ(𝒢 ∣ Θ) = ∑
(s,p,o)∈𝒢
log σ (fp(es, eo))+ ∑
(s,p,o)∉𝒢
log [1 − σ (fp(es, eo))]Washington
Malia Ann Obama
Sasha Obama
Barack Obama
Michelle Obama
lives in
parent of
?
P(BO married MO) ∝
fmarried( , )
Neural Link Prediction
![Page 18: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/18.jpg)
Neural Link Prediction — Scoring Functions
Models Scoring Functions Parameters
RESCAL [Nickel et al. 2011]
TransE [Bordes et al. 2013]
DistMult [Yang et al. 2015]
HolE [Nickel et al. 2016]
ComplEx [Trouillon et al. 2016]
ConvE [Dettmers et al. 2017]
− es + rp − eo2
p
⟨es, rp, eo⟩
Re (⟨es, rp, eo⟩)
r⊤p (ℱ−1 [ℱ[es] ⊙ ℱ[eo]])
f (vec (f ([es; rp] * ω)) W) eo
rp ∈ ℝk
rp ∈ ℝk
rp ∈ ℝk
rp ∈ ℝk, W ∈ ℝc×k
rp ∈ ℂk
e⊤s Wpeo Wp ∈ ℝk×k
The interaction between the latent features is defined by the scoring function — several variants in the literature:f( ⋅ )
![Page 19: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/19.jpg)
Neural Link Prediction — Scoring Functions
Models Scoring Functions Parameters
RESCAL [Nickel et al. 2011]
TransE [Bordes et al. 2013]
DistMult [Yang et al. 2015]
HolE [Nickel et al. 2016]
ComplEx [Trouillon et al. 2016]
ConvE [Dettmers et al. 2017]
− es + rp − eo2
p
⟨es, rp, eo⟩
Re (⟨es, rp, eo⟩)
r⊤p (ℱ−1 [ℱ[es] ⊙ ℱ[eo]])
f (vec (f ([es; rp] * ω)) W) eo
rp ∈ ℝk
rp ∈ ℝk
rp ∈ ℝk
rp ∈ ℝk, W ∈ ℝc×k
rp ∈ ℂk
e⊤s Wpeo Wp ∈ ℝk×k
The interaction between the latent features is defined by the scoring function — several variants in the literature:f( ⋅ )
![Page 20: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/20.jpg)
Neural Link Prediction — Scoring Functions
Models Scoring Functions Parameters
RESCAL [Nickel et al. 2011]
TransE [Bordes et al. 2013]
DistMult [Yang et al. 2015]
HolE [Nickel et al. 2016]
ComplEx [Trouillon et al. 2016]
ConvE [Dettmers et al. 2017]
− es + rp − eo2
p
⟨es, rp, eo⟩
Re (⟨es, rp, eo⟩)
r⊤p (ℱ−1 [ℱ[es] ⊙ ℱ[eo]])
f (vec (f ([es; rp] * ω)) W) eo
rp ∈ ℝk
rp ∈ ℝk
rp ∈ ℝk
rp ∈ ℝk, W ∈ ℝc×k
rp ∈ ℂk
e⊤s Wpeo Wp ∈ ℝk×k
The interaction between the latent features is defined by the scoring function — several variants in the literature:f( ⋅ )
![Page 21: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/21.jpg)
Neural Link Prediction — Scoring Functions
Models Scoring Functions Parameters
RESCAL [Nickel et al. 2011]
TransE [Bordes et al. 2013]
DistMult [Yang et al. 2015]
HolE [Nickel et al. 2016]
ComplEx [Trouillon et al. 2016]
ConvE [Dettmers et al. 2017]
− es + rp − eo2
p
⟨es, rp, eo⟩
Re (⟨es, rp, eo⟩)
r⊤p (ℱ−1 [ℱ[es] ⊙ ℱ[eo]])
f (vec (f ([es; rp] * ω)) W) eo
rp ∈ ℝk
rp ∈ ℝk
rp ∈ ℝk
rp ∈ ℝk, W ∈ ℝc×k
rp ∈ ℂk
e⊤s Wpeo Wp ∈ ℝk×k
The interaction between the latent features is defined by the scoring function — several variants in the literature:f( ⋅ )
![Page 22: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/22.jpg)
Evaluation Metrics — Area Under the Precision-Recall Curve (AUC-PR), Mean Reciprocal Rank (MRR), Hits@k. In MRR and Hits@k, for each test triple:
• Modify its subject with all the entities in the Knowledge Graph,• Score all the triple variants, and compute the rank of the original test triple,• Repeat for the object.
MRR =1
|𝒯 |
|𝒯|
∑i=1
1ranki
, HITS@k =|{ranki ≤ 10} |
|𝒯 |From [Lacroix et al. ICML 2018]
Neural Link Prediction — Accuracy
![Page 23: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/23.jpg)
Convolutional 2D Knowledge Graph Embeddings
Subject Embedding
Predicate Embedding
[AAAI 2018]
Idea — use ideas from computer vision for modeling the interactions between latent features.
![Page 24: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/24.jpg)
Convolutional 2D Knowledge Graph Embeddings
Subject Embedding
Predicate Embedding
Reshape
“Latent Images”
Idea — use ideas from computer vision for modeling the interactions between latent features.
[AAAI 2018]
![Page 25: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/25.jpg)
Convolutional 2D Knowledge Graph Embeddings
Subject Embedding
Predicate Embedding
Reshape
“Latent Images”
2D ConvolutionsConcatenate
Idea — use ideas from computer vision for modeling the interactions between latent features.
[AAAI 2018]
![Page 26: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/26.jpg)
Convolutional 2D Knowledge Graph Embeddings
Subject Embedding
Predicate Embedding
Reshape
“Latent Images”
2D ConvolutionsConcatenateProjection Logits
Product with Entity Matrix
Idea — use ideas from computer vision for modeling the interactions between latent features.
[AAAI 2018]✓Scalable
✓State-of-the-art Results
![Page 27: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/27.jpg)
Convolutional 2D Knowledge Graph EmbeddingsIdea — use ideas from computer vision for modeling the interactions between latent features.
[AAAI 2018]
0.1
0.225
0.35
0.475
0.6
DistMult ComplEx R-GCN ConvE
MRR Hits@1 Hits@3 Hits@10
✓Efficiency via parameter sharing ✓State-of-the-art Results
![Page 28: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/28.jpg)
Interpreting Knowledge Graph EmbeddingsQuite hard to understand the semantics of the learned representations..
[Minervini et al. ECML 2017]
![Page 29: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/29.jpg)
Interpreting Knowledge Graph EmbeddingsQuite hard to understand the semantics of the learned representations..
.. but we can use their geometric relationships for identifying — and incorporating — semantic relationships between them.
[Minervini et al. ECML 2017]
![Page 30: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/30.jpg)
Regularising Knowledge Graph EmbeddingsQuite hard to understand the semantics of the learned representations..
.. but we can use their geometric relationships for identifying — and incorporating — semantic relationships between them.
[Minervini et al. ECML 2017]
![Page 31: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/31.jpg)
Regularising Knowledge Graph EmbeddingsQuite hard to understand the semantics of the learned representations..
.. but we can use their geometric relationships for identifying — and incorporating — semantic relationships between them.
[Minervini et al. ECML 2017]
✕ is a(x, y) ∧ is a(y, z) ⇒ is a(x, z)
![Page 32: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/32.jpg)
Incorporating Background Knowledge via Adversarial Training
Idea — adversarial training process where, iteratively:
[Minervini et al. UAI 2017]
![Page 33: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/33.jpg)
Incorporating Background Knowledge via Adversarial Training
Idea — adversarial training process where, iteratively: • An adversary searches for inputs where the model violates constraints
[Minervini et al. UAI 2017]
e.g. x, y, z such that is a(x, y) ∧ is a(y, z) ∧ ¬is a(x, z)
![Page 34: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/34.jpg)
Incorporating Background Knowledge via Adversarial Training
Idea — adversarial training process where, iteratively: • An adversary searches for inputs where the model violates constraints • The model is regularised to correct such violations.
[Minervini et al. UAI 2017]
![Page 35: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/35.jpg)
Incorporating Background Knowledge via Adversarial Training
Idea — adversarial training process where, iteratively: • An adversary searches for inputs where the model violates constraints, • The model is regularised to correct such violations.
Formally:
minΘ
ℒdata(D ∣ Θ) + λ maxS
ℒviolation(S, D ∣ Θ)
[Minervini et al. UAI 2017]
e.g. S = {x, y, z} such that is a(x, y) ∧ is a(y, z) ∧ ¬is a(x, z)
![Page 36: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/36.jpg)
Incorporating Background Knowledge via Adversarial Training
Idea — adversarial training process where, iteratively: • An adversary searches for inputs where the model violates constraints, • The model is regularised to correct such violations.
Formally:
• Inputs S can be either input space or embedding space
• In most interesting cases, max has closed form solutions
• Constraints are guaranteed to hold everywhere in embedding space.
[Minervini et al. UAI 2017]
minΘ
ℒdata(D ∣ Θ) + λ maxS
ℒviolation(S, D ∣ Θ)
e.g. S = {ex, ey, ez} such that is a(ex, ey) ∧ is a(ey, ez) ∧ ¬is a(ex, ez)
![Page 37: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/37.jpg)
Incorporating Background Knowledge via Adversarial Training
Idea — adversarial training process where, iteratively: • An adversary searches for inputs where the model violates constraints, • The model is regularised to correct such violations.
Formally:
[Minervini et al. UAI 2017]
e.g. S = {ex, ey, ez} such that is a(ex, ey) ∧ is a(ey, ez) ∧ ¬is a(ex, ez)
✓Incorporates Background Knowledge
✓Verifiable
minΘ
ℒdata(D ∣ Θ) + λ maxS
ℒviolation(S, D ∣ Θ)
![Page 38: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/38.jpg)
Incorporating Background Knowledge via Adversarial Training
[Minervini et al. UAI 2017]
55
60.75
66.5
72.25
78
TransE KALE-Pre KALE-Joint DistMult ASR-DistMult ComplEx ASR-ComplEx
Hits@3 Hits@5 Hits@10
![Page 39: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/39.jpg)
Incorporating Background Knowledge via Adversarial Training
[Minervini et al. UAI 2017]
55
60.75
66.5
72.25
78
TransE KALE-Pre KALE-Joint DistMult ASR-DistMult ComplEx ASR-ComplEx
Hits@3 Hits@5 Hits@10
![Page 40: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/40.jpg)
Incorporating Background Knowledge in Natural Language Inference Models
[Minervini et al. CoNLL 2018]
Natural Language Inference — detect the type of relationship, i.e. entailment, contradiction, neutral, between two sentences.
![Page 41: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/41.jpg)
Incorporating Background Knowledge in Natural Language Inference Models
If a stentence x contradicts y, then also y contradicts x. If x entails y, and y entails z, then x also entails z.
[Minervini et al. CoNLL 2018]
Natural Language Inference — detect the type of relationship, i.e. entailment, contradiction, neutral, between two sentences.
![Page 42: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/42.jpg)
Incorporating Background Knowledge in Natural Language Inference Models
If a stentence x contradicts y, then also y contradicts x. If x entails y, and y entails z, then x also entails z.
x) A man in uniform is pushing a medical bed. y) A man is pushing carrying something.
[Minervini et al. CoNLL 2018]
Natural Language Inference — detect the type of relationship, i.e. entailment, contradiction, neutral, between two sentences.
![Page 43: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/43.jpg)
Incorporating Background Knowledge in Natural Language Inference Models
If a stentence x contradicts y, then also y contradicts x. If x entails y, and y entails z, then x also entails z.
ℒviolation({x, y}) : 0.01 ⇝ 0.92P(x entails y) = 0.72
P(y contradicts x) = 0.93
x) A man in uniform is pushing a medical bed. y) A man is pushing carrying something.
[Minervini et al. CoNLL 2018]
Natural Language Inference — detect the type of relationship, i.e. entailment, contradiction, neutral, between two sentences.
![Page 44: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/44.jpg)
Incorporating Background Knowledge in Natural Language Inference Models
[Minervini et al. CoNLL 2018]
![Page 45: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/45.jpg)
End-to-End Differentiable Reasoning
![Page 46: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/46.jpg)
End-to-End Differentiable ReasoningCore idea — we can combine neural networks and symbolic models by re-implementing classic reasoning algorithms using end-to-end differentiable (neural) architectures.
(Black-Box) Neural Models
•Can generalise from noisy and ambiguois modalities
•Can learn representations from data•SOTA on a number of tasks
Symbolic Reasoning Models
•Data efficient•Interpretable•Explainable•Verifiable•Can incorporate background
knowledge and constraints
![Page 47: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/47.jpg)
Reasoning via Backward Chaining
Backward Chaining — start with a list of goals, and work backwards from the consequent Q to the antecedent P to see if any data supports any of the consequents.
q(X) ← p(X)q(a)?p(a)
p(b)p(c)
…
You can see backward chaining as a query reformulation strategy.
![Page 48: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/48.jpg)
Reasoning via Backward Chaining
Backward Chaining — start with a list of goals, and work backwards from the consequent Q to the antecedent P to see if any data supports any of the consequents.
q(X) ← p(X)q(a)?p(a)
p(b)p(c)
…
You can see backward chaining as a query reformulation strategy.
p(a)
![Page 49: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/49.jpg)
Reasoning via Backward Chaining
Backward Chaining — start with a list of goals, and work backwards from the consequent Q to the antecedent P to see if any data supports any of the consequents.
q(X) ← p(X)q(a)?p(a)
p(b)p(c)
…
You can see backward chaining as a query reformulation strategy.
✓
p(a)
![Page 50: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/50.jpg)
End-to-End Differentiable Reasoning𝚐𝚛𝚊𝚗𝚍𝙿𝚊𝙾𝚏 (𝚊𝚋𝚎, 𝚋𝚊𝚛𝚝)
𝚐𝚛𝚊𝚗𝚍𝙵𝚊𝚝𝚑𝚎𝚛𝙾𝚏 (𝚊𝚋𝚎, 𝚋𝚊𝚛𝚝)
✓ ✓✓
sim = 1sim = 1sim = 0.9
![Page 51: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/51.jpg)
End-to-End Differentiable ReasoningKnowledge Base:
𝚐𝚛𝚊𝚗𝚍𝙿𝚊𝙾𝚏(𝚊𝚋𝚎, 𝚋𝚊𝚛𝚝)
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛)
proof score S1
𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(𝚑𝚘𝚖𝚎𝚛, 𝚋𝚊𝚛𝚝)
proof score S2
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛)𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(𝚑𝚘𝚖𝚎𝚛, 𝚋𝚊𝚛𝚝)
𝚐𝚛𝚊𝚗𝚍𝙵𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Y ) ⇐𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Z ),𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(Z, Y ) .
![Page 52: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/52.jpg)
End-to-End Differentiable ReasoningKnowledge Base:
𝚐𝚛𝚊𝚗𝚍𝙿𝚊𝙾𝚏(𝚊𝚋𝚎, 𝚋𝚊𝚛𝚝)
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛)
proof score S1
𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(𝚑𝚘𝚖𝚎𝚛, 𝚋𝚊𝚛𝚝)
proof score S2
𝚐𝚛𝚊𝚗𝚍𝙵𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Y )X /𝚊𝚋𝚎 Y/𝚋𝚊𝚛𝚝
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, Z )𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(Z, 𝚋𝚊𝚛𝚝)
Subgoals:
proof score S3
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛)𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(𝚑𝚘𝚖𝚎𝚛, 𝚋𝚊𝚛𝚝)
𝚐𝚛𝚊𝚗𝚍𝙵𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Y ) ⇐𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Z ),𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(Z, Y ) .
![Page 53: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/53.jpg)
End-to-End Differentiable ReasoningKnowledge Base:
𝚐𝚛𝚊𝚗𝚍𝙿𝚊𝙾𝚏(𝚊𝚋𝚎, 𝚋𝚊𝚛𝚝)
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛)
proof score S1
𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(𝚑𝚘𝚖𝚎𝚛, 𝚋𝚊𝚛𝚝)
proof score S2
𝚐𝚛𝚊𝚗𝚍𝙵𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Y )X /𝚊𝚋𝚎 Y/𝚋𝚊𝚛𝚝
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, Z )𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(Z, 𝚋𝚊𝚛𝚝)
Subgoals:
proof score S3
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, Z )Z
proof score S4
proof score S5
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛) …
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛)𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(𝚑𝚘𝚖𝚎𝚛, 𝚋𝚊𝚛𝚝)
𝚐𝚛𝚊𝚗𝚍𝙵𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Y ) ⇐𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Z ),𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(Z, Y ) .
![Page 54: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/54.jpg)
End-to-End Differentiable ReasoningKnowledge Base:
𝚐𝚛𝚊𝚗𝚍𝙿𝚊𝙾𝚏(𝚊𝚋𝚎, 𝚋𝚊𝚛𝚝)
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛)
proof score S1
𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(𝚑𝚘𝚖𝚎𝚛, 𝚋𝚊𝚛𝚝)
proof score S2
𝚐𝚛𝚊𝚗𝚍𝙵𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Y )X /𝚊𝚋𝚎 Y/𝚋𝚊𝚛𝚝
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, Z )𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(Z, 𝚋𝚊𝚛𝚝)
Subgoals:
proof score S3
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, Z )Z
proof score S4
proof score S5
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛) …∑F∈K
log pKB∖F(F)
− ∑F̃∼corr(F)
log pKB(F̃)
Train viaSelf-Supervision:
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛)𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(𝚑𝚘𝚖𝚎𝚛, 𝚋𝚊𝚛𝚝)θ1(X, Y ) ⇐ θ2(X, Z ), θ3(Z, Y ) .
![Page 55: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/55.jpg)
End-to-End Differentiable ReasoningKnowledge Base:
𝚐𝚛𝚊𝚗𝚍𝙿𝚊𝙾𝚏(𝚊𝚋𝚎, 𝚋𝚊𝚛𝚝)
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛)
proof score S1
𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(𝚑𝚘𝚖𝚎𝚛, 𝚋𝚊𝚛𝚝)
proof score S2
𝚐𝚛𝚊𝚗𝚍𝙵𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Y )X /𝚊𝚋𝚎 Y/𝚋𝚊𝚛𝚝
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, Z )𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(Z, 𝚋𝚊𝚛𝚝)
Subgoals:
proof score S3
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, Z )Z
proof score S4
proof score S5
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛) …
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛)𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(𝚑𝚘𝚖𝚎𝚛, 𝚋𝚊𝚛𝚝)
𝚐𝚛𝚊𝚗𝚍𝙵𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Y ) ⇐𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Z ),𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(Z, Y ) .
![Page 56: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/56.jpg)
End-to-End Differentiable ReasoningKnowledge Base:
𝚐𝚛𝚊𝚗𝚍𝙿𝚊𝙾𝚏(𝚊𝚋𝚎, 𝚋𝚊𝚛𝚝)
𝚐𝚛𝚊𝚗𝚍𝙵𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Y )X /𝚊𝚋𝚎 Y/𝚋𝚊𝚛𝚝
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, Z )𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(Z, 𝚋𝚊𝚛𝚝)
Subgoals:
proof score S3
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, Z )Z
proof score S4
proof score S5
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛) …
𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(𝚊𝚋𝚎, 𝚑𝚘𝚖𝚎𝚛)𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(𝚑𝚘𝚖𝚎𝚛, 𝚋𝚊𝚛𝚝)
𝚐𝚛𝚊𝚗𝚍𝙵𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Y ) ⇐𝚏𝚊𝚝𝚑𝚎𝚛𝙾𝚏(X, Z ),𝚙𝚊𝚛𝚎𝚗𝚝𝙾𝚏(Z, Y ) .
[Minervini et al. 2018, AAAI 2020, Welbl et al. ACL 2019]
![Page 57: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/57.jpg)
End-to-End Differentiable Reasoning
[Minervini et al. AAAI 2020]
![Page 58: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/58.jpg)
End-to-End Differentiable Reasoning
[Minervini et al. AAAI 2020]
![Page 59: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/59.jpg)
End-to-End Differentiable Reasoning with Natural Language
We can embed facts from the KG and facts from text in a shared embedding space, and learn to reason over them jointly:
5XOH�*URXS�S�;��<�����T�<��;� 5XOHV 5XOH�*URXS�S�;��<�����T�;��=���U�=��<�
; < �� < ;
HQFRGHU
.%
HQFRGHU
4XHU\
$1'
FRQWDLQHG,Q�5LYHU�7KDPHV��8.�����
³/RQGRQ�LV�ORFDWHG�LQ�WKH�8.´
³/RQGRQ�LV�VWDQGLQJ�RQ�WKH�5LYHU�7KDPHV´
³>;@�LV�ORFDWHG�LQ�WKH�><@´�;��<�����ORFDWHG,Q�;��<�
ORFDWHG,Q�;��<�����ORFDWHG,Q�;��=���ORFDWHG,Q�=��<�
.%�5HS�� 7H[W�5HSUHVHQWDWLRQV
; < �� < ;; < �� < ; ; < �� ; =�
= <; < �� ; =�
= <
5HFXUVH
N�11�25
[Minervini et al. AAAI 2020]
![Page 60: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/60.jpg)
End-to-End Differentiable Reasoning with Natural Language
We can embed facts from the KG and facts from text in a shared embedding space, and learn to reason over them jointly:
[Welbl et al. ACL 2019, Minervini et al. AAAI 2020]
![Page 61: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/61.jpg)
End-to-End Differentiable Reasoning with Natural Language
We can embed facts from the KG and facts from text in a shared embedding space, and learn to reason over them jointly:
[Welbl et al. ACL 2019, Minervini et al. AAAI 2020]
![Page 62: Explainable, Data-Efficient, Verifiable Representation ...kc2813/xaiseminar/... · Barack Obama is married to Michelle Obama Michelle Obama is a Lawyer Michelle Obama lives in United](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f3b1fd0fbae0a4737f54d/html5/thumbnails/62.jpg)
Thank you!