Visual Information Systems Image Content. Description of Content – image processing Primitive image properties Primitive image properties Through image

Download Visual Information Systems Image Content. Description of Content – image processing Primitive image properties Primitive image properties Through image

Post on 06-Jan-2018

219 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

Visual Information Systems Image Content Description of Content image processing Primitive image properties Primitive image properties Through image processing techniques Through image processing techniques Colour, texture, local shape Colour, texture, local shape The need of combination of these properties into a consistent set of localised properties The need of combination of these properties into a consistent set of localised properties There can be weighting scheme to balance the importance of each type of property. There can be weighting scheme to balance the importance of each type of property. Image features Image features Integration of primitive properties a separation between color, local geometry, and texture. a separation between color, local geometry, and texture. an integrated view on color, texture, and local geometry is urgently needed as only an integrated view on local properties can provide the means to distinguish among hundreds of thousands different images. an integrated view on color, texture, and local geometry is urgently needed as only an integrated view on local properties can provide the means to distinguish among hundreds of thousands different images. Further research is needed in the design of complete sets of image properties with well- described variant conditions which they are capable of handling. Further research is needed in the design of complete sets of image properties with well- described variant conditions which they are capable of handling. Image features Grouping Data, Global and Accumulating Features, Salient Features, Signs, Shape and Object Features, Description of Structure and Lay-Out Grouping Data, Global and Accumulating Features, Salient Features, Signs, Shape and Object Features, Description of Structure and Lay-Out Also in the description of the image by features, it should be kept in mind that for retrieval a total understanding of the image is rarely needed. Also in the description of the image by features, it should be kept in mind that for retrieval a total understanding of the image is rarely needed. the deeper one goes into the semantics of the pictures, the deeper the understanding of the picture will also have to be the deeper one goes into the semantics of the pictures, the deeper the understanding of the picture will also have to be With segmentation With segmentation no segmentation no segmentation Interpretation And Similarity Measure Semantic features aim at encoding interpretations of the image which may be relevant to the application. Semantic features aim at encoding interpretations of the image which may be relevant to the application. feature set can be explained feature set can be explained derives an unilateral interpretation from the feature set derives an unilateral interpretation from the feature set compares the feature set with the elements in a given data set on the basis of a similarity function compares the feature set with the elements in a given data set on the basis of a similarity function In content-based retrieval, it is useful to push the semantic interpretation of features derived from the image as far as one can. In content-based retrieval, it is useful to push the semantic interpretation of features derived from the image as far as one can. Similarity Measurement A different road to assigning a meaning to an observed feature set, is to compare a pair of observations by a similarity function. a kind of interpretation A different road to assigning a meaning to an observed feature set, is to compare a pair of observations by a similarity function. a kind of interpretation And this is the advantage to have content- based retrieval. And this is the advantage to have content- based retrieval. Semantic Similarity knowledge-based type abstraction hierarchies knowledge-based type abstraction hierarchies concept-space concept-space a linguistic description of texture patch visual qualities is given and ordered in a hierarchy of perceptual importance on the basis of extensive psychological experimentation. a linguistic description of texture patch visual qualities is given and ordered in a hierarchy of perceptual importance on the basis of extensive psychological experimentation. A more general concept of similarity is needed for relevance feedback, in which similarity with respect to an ensemble of images is required. A more general concept of similarity is needed for relevance feedback, in which similarity with respect to an ensemble of images is required. Different Levels of Content-base Indexing and Retrieval syntactical level: mainly deal with colour, shape, texture etc. Some used manual annotation to index data syntactical level: mainly deal with colour, shape, texture etc. Some used manual annotation to index data e.g. retrieval system let users to fill forms to provide queries, like location, colour etc categories, like the work done in Berkeley e.g. retrieval system let users to fill forms to provide queries, like location, colour etc categories, like the work done in Berkeley semantic level : semantic level : analyse captions analyse captions purely used text information and didnt make use of the information inherent in the images purely used text information and didnt make use of the information inherent in the images complicated algorithm applied on small scale complicated algorithm applied on small scale Multi-level indexing An advantage of image indexing based on multi-level contents rather than solely on low-level features such as texture and colours, is that it would readily provide the basic framework required for "semantic interoperability" when one tries to search through, not only one, but a federation of image collections from different disciplines. An advantage of image indexing based on multi-level contents rather than solely on low-level features such as texture and colours, is that it would readily provide the basic framework required for "semantic interoperability" when one tries to search through, not only one, but a federation of image collections from different disciplines. Learning from Feedback The interacting user brings about many new challenges for the response time of the system. The interacting user brings about many new challenges for the response time of the system. Content-based image retrieval is only scalable to large data sets when the database is able to anticipate what interactive queries will be made. Content-based image retrieval is only scalable to large data sets when the database is able to anticipate what interactive queries will be made. A frequent assumption is that the image set, the features, and the similarity function are known in advance. In a truly interactive session, the assumptions are no longer valid. A frequent assumption is that the image set, the features, and the similarity function are known in advance. In a truly interactive session, the assumptions are no longer valid. A change from static to dynamic indexing is required. (Arnold 2000) A change from static to dynamic indexing is required. (Arnold 2000) An integrated issue It will demand its own view of things as it is our belief that content-based retrieval in the end will not be part of the field of computer vision alone. The man-machine interface, domain knowledge, and database technology each will have their impact on the product. It will demand its own view of things as it is our belief that content-based retrieval in the end will not be part of the field of computer vision alone. The man-machine interface, domain knowledge, and database technology each will have their impact on the product. Summary The heritage of computer vision. The heritage of computer vision. The influence on computer vision. The influence on computer vision. Deal with large data sets. Deal with large data sets. the absence of a general method for strong segmentation. the absence of a general method for strong segmentation. has revitalized interest in color image processing. has revitalized interest in color image processing. attention for invariance has been revitalized attention for invariance has been revitalized Similarity and learning. Similarity and learning. Interaction. Interaction. The need for databases. The need for databases. The problem of evaluation. The problem of evaluation. The semantic gap and other sources. The semantic gap and other sources. Colour Colour is a visual feature which is immediately perceived when looking at an image. Colour is a visual feature which is immediately perceived when looking at an image. The recorded colour varies considerably with the orientation of the surface, the viewpoint of the camera, the position of the illumination, the spectrum of the illuminant, and the way the light interacts with the object. The recorded colour varies considerably with the orientation of the surface, the viewpoint of the camera, the position of the illumination, the spectrum of the illuminant, and the way the light interacts with the object. The human perception of colour is an intricate topic where many attempts have been made to capture perceptual similarity. The human perception of colour is an intricate topic where many attempts have been made to capture perceptual similarity. Colour Light is energy, specifically electromagnetic energy eye (energy detector) Light is energy, specifically electromagnetic energy eye (energy detector) The eye can distinguish between some types of electromagnetic energy. Those distinctions are seen as colours. The eye can distinguish between some types of electromagnetic energy. Those distinctions are seen as colours. The whiteness of an image area and the amount of light hitting the eye The whiteness of an image area and the amount of light hitting the eye The actual reflectance The actual reflectance The brightness of incident light The brightness of incident light The incidence angle: the angle at which the light hits the object (one per light source, and there may