a linear-algebraic technique with an application in semantic image retrieval
DESCRIPTION
A Linear-Algebraic Technique with an Application in Semantic Image Retrieval. Jonathon S. Hare, Paul H. Lewis 2006,CIVR. Outline. Introduction Using Linear-Algebra to Associate Image and Terms Method Decomposing the Observation Matrix Using the Terms as a Basis for New Documents - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/1.jpg)
1
A Linear-Algebraic Technique with an Application in Semantic Image Retrieval
Jonathon S. Hare, Paul H. Lewis2006,CIVR
![Page 2: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/2.jpg)
2
Outline
• Introduction• Using Linear-Algebra to Associate
Image and Terms• Method– Decomposing the Observation Matrix– Using the Terms as a Basis for New Documents
• Experimental Result
![Page 3: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/3.jpg)
3
Introduction
• Automatic annotation of image
![Page 4: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/4.jpg)
4
Outline
• Introduction• Using Linear-Algebra to Associate
Image and Terms• Method– Decomposing the Observation Matrix– Using the Terms as a Basis for New Documents
• Experimental Result
![Page 5: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/5.jpg)
5
Using Linear-Algebra to Associate Image and Terms
• Any document can be described by a series of observations , or measurements.
• We refer to each of these observations as terms.
• Don’t explicitly annotate images.
![Page 6: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/6.jpg)
6
Using Linear-Algebra to Associate Image and Terms
DTO *
Given a corpus of n documents, it is possible to form a matrix of m observations or measurements (i.e. a term-document matrix). This m × n observation matrix, O, essentially represents a combination of terms and documents, and can be factored into a separate term matrix, T, and document matrix, D:
![Page 7: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/7.jpg)
7
Outline
• Introduction• Using Linear-Algebra to Associate
Image and Terms• Method– Decomposing the Observation Matrix– Using the Terms as a Basis for New Documents
• Experimental Result
![Page 8: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/8.jpg)
8
Decomposing the Observation Matrix
• Lemma 1(The rank principle for a noise-free term-document matrix)
Without noise, the observation matrix ,O, has a rank at most equal to the number of independent terms or documents observed.
• In reality, the observation term-document matrix is not at all noise free.
![Page 9: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/9.jpg)
9
Decomposing the Observation Matrix
• The observation matrix, O can be decomposed using SVD into a m × r matrix U,a r × r diagonal matrix Σ and a r × n matrix VT , O = UΣVT , such that UTU = VVT = VTV = I, where I is the identity matrix.
![Page 10: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/10.jpg)
10
Decomposing the Observation Matrix
• Assume O∗ is the ideal, noise-free observation matrix, with k independent terms. Now partitioning the U, Σ and VT matrices as follows:
TKKK
TNNN
TKKK
T
VUO
VUVUVUO
*
![Page 11: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/11.jpg)
11
Decomposing the Observation Matrix
• Lemma 2 (The rank principle for a noisy term-document matrix).
All of the information about the terms and documents in O is encoded in its k largest singular values together with the corresponding left and right eigenvectors.
![Page 12: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/12.jpg)
12
Decomposing the Observation Matrix
• define the estimated noise-free term matrix, ,and document matrix, .• ,where represents the estimated
noise-free observation matrix.
k
def
UT ˆ TKK
def
VD ˆ
DTO ˆˆˆ
![Page 13: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/13.jpg)
13
Decomposing the Observation Matrix
• The rows of the term matrix create a basis representing a position in the space of each of the observed terms.
• The columns of the document matrix represent positions of the observed documents in the space.
![Page 14: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/14.jpg)
14
Outline
• Introduction• Using Linear-Algebra to Associate
Image and Terms• Method– Decomposing the Observation Matrix– Using the Terms as a Basis for New Documents
• Experimental Result
![Page 15: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/15.jpg)
15
Using the Terms as a Basis for New Documents
• Theorem 1 (Projection of partially observed measurements). The term-matrix of a decomposed fully-observed measurement matrix can be used to project a partially observed measurement matrix into a document matrix that encapsulates estimates of the unobserved terms.
DTP ˆˆ
![Page 16: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/16.jpg)
16
Using the Terms as a Basis for New Documents
• To project a new partially observed measurement matrix into a basis created from a fully observed training matrix, we need only pre-multiply the new observation matrix by the transpose of the training term matrix.
![Page 17: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/17.jpg)
17
Using the Terms as a Basis for New Documents
• In order to query the document set for documents relevant to a term, we just need to rank all of the documents based on their position in the space with respect to the position of the query term in the space.
• The cosine similarity is a suitable measure for this task.
![Page 18: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/18.jpg)
18
Outline
• Introduction• Using Linear-Algebra to Associate
Image and Terms• Method– Decomposing the Observation Matrix– Using the Terms as a Basis for New Documents
• Experimental Result
![Page 19: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/19.jpg)
19
Experimental Result
![Page 20: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/20.jpg)
20
Experimental Result
• we chose a value of k = 100 for the followingexperiments.
![Page 21: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/21.jpg)
21
Experimental Result
![Page 22: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/22.jpg)
22
Experimental Result
![Page 23: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/23.jpg)
23
Experimental Result
![Page 24: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/24.jpg)
24
Experimental Result• we use a much simpler feature: a 64-bin
global RGB histogram.
![Page 25: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/25.jpg)
25
Experimental Result
![Page 26: A Linear-Algebraic Technique with an Application in Semantic Image Retrieval](https://reader035.vdocuments.site/reader035/viewer/2022081604/56815e4f550346895dccc9c5/html5/thumbnails/26.jpg)
26
Conclusions
• This paper presented a novel approach.• The method does not produce equal
performance for all queries.– The features are not enough.– The training data are not enough.