lateral reference transformation for emergenceusers.monash.edu/~srini/theses/viranga_thesis.pdf ·...

Lateral Reference Transformation for Emergence

by

Arvind Viranga Ratnaike

Thesis

Submitted by Arvind Viranga Ratnaike

for fulfillment of the Requirements for the Degree of

Doctor of Philosophy

Faculty of Information Technology

Monash University

June, 2009

c© Copyright

by


2009

To Samar, Shahaan, Srini and Surya

iii


Declaration

I declare that this thesis is my own work and has not been submitted in anyform for another degree or diploma at any university or other institute of tertiaryeducation. Information derived from the published and unpublished work of othershas been acknowledged in the text and a list of references is given.

Arvind Viranga RatnaikeJune 28, 2009

iv

Acknowledgments

I am thankful to everyone who has advised, argued with, put up with, and fed me

during my somewhat obsessive journey through my thesis.

I’m especially thankful to my parents Sujiva and Tissa Ratnaike, who have been

there for me throughout my life. They are in no small part to be thanked for keeping

me sane through this and many other aspects of my life.

Four people in particular have been subjected to conversations about my thesis.

They are my office mates at Monash: Samar Zutshi and Shahaan Ayyub, and my

two supervisors Bala Srinivasan and Surya Nepal. Samar and Srini also proofread

my dissertation. I would like also to thank my former supervisor Leila Alem, who

was my supervisor when my research area was very different.

Also the people from the postgraduate forum in the Department of Computer

Science and Software Engineering. With particular thanks to the postgraduate

coordinator David Squire who has always been approachable and helpful. And to

Lawrence Bull whose LaTeX editing environment I continue to use.

I thank the departmental staff who have always been quite friendly. There are

two staff I want to thank especially. Michelle Ketchen, the very busy department

manager, who always found nice places for me to study. Alison Mitchell, the most

helpful and professional administrator I have thus far encountered in my life.

Thank you to the applied-linguists Christiane Momberg and Giao Tran, who

helped me to understand concepts in linguistics. Also, thank you to the radiologist

John DeCampo for explaining the basics of radiology.

There was a haven in which I occasionally sought refuge from my thesis: the

home of Mingfang Wu, Mingwei Zhou and Kathy Zhou.


Monash University

June 2009

v


Abstract

This thesis investigates transformation of forms of indication; a transition from data

in one form to a description in another form. The data input indicates a situation;

the description output indicates a situation. Though both should, ideally, indicate

the same situation, there is likely to be a drift in what situation is indicated. A

theory for transformation by emergence is introduced, as is a framework for the

transformation. The framework can transform an available known form of reference

to a more accessible, destination form. This destination form is also the form of a

domain knowledge source.

This kind of reference transformation is a specialization of a more general prob-

lem: that of improving indication; using possibly multiple source forms resulting

in possibly multiple destination forms. The improvement should outweigh the

cumulative error incurred during the act of improvement. In general, automatic

interpretation of multimedia or sensory cues remains a challenge. Synthesis of

interpretation is partly completed in the mind of the recipient, where association

with prior experience, of both notions and indication thereof, takes place. The scope

of the thesis is constrained to what a machine can manipulate. Lateral reference

transformation should occur during early emergence. The transformation results in a

collection of partial descriptions, which can be used to compose a greater situational

description.

The reference transformation framework is based on an underlying philosophy of

emergence. Situational descriptions emerge from the interaction of domain knowl-

edge and situational data. An assumption is made that data and knowledge elements

vi

can interact. The reference transformation is lateral, in the sense that the principal

quality being increased by emergence is clarity rather than complexity.

Domain knowledge is converted to a modifiable internal entity known as a bridg-

ing entity. The framework is designed to be independent of media types. The

framework adopts the destination form as the basis of the internal form of the

bridging entity. It is the bridging entity that is gradually manipulated rather than

the media of the original situational data indicators or references. Interaction of

the bridging entity with situational data, creates a perturbed bridging entity. The

bridging entity can be progressively perturbed through interaction with further data,

resulting in an entity that is acceptable as a description of the situation. The

final bridging entity is mapped back to an accessible knowledge form, based on an

auxiliary entity that stores system experience. Confidence in the acceptability of

transformation is based on how much of the situation is considered familiar.

Context is considered from two perspectives. From the perspective of the data,

it is the knowledge and the experience available for interpretation. This includes

entities and concepts that aren’t directly observable in the data. From the perspec-

tive of the description, context is whatever influences the emergence and subsequent

interpretation, without being explicitly indicated by the description.

The framework is investigated by considering its various aspects, in the contexts

of disparate situational domains. The conclusions, based on this investigation, lead

to modifications to the initial theory and framework.

vii

Contents

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Different Ways of Indicating the Same Idea . . . . . . . . . . . . . . . 2

1.1.1 Indication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Emergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.3 Motivation and Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.3.1 Making Sense of Non-text Media Content . . . . . . . . . . . . 10

1.3.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.3.3 Structure of the Dissertation . . . . . . . . . . . . . . . . . . . 14

2 Interpretation of Situational Data . . . . . . . . . . . . . . . . . . . 16

2.1 Indication in Non-Text Data . . . . . . . . . . . . . . . . . . . . . . . 17

2.1.1 Semantics and Meaning . . . . . . . . . . . . . . . . . . . . . 19

2.1.2 Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.1.3 Communication . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.2 Emergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.2.1 Classical and Computational Emergence . . . . . . . . . . . . 30

2.2.2 Emergent Semantics . . . . . . . . . . . . . . . . . . . . . . . 32

2.2.3 Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.2.4 Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.2.5 Self-organization . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.3 Reference Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.3.1 Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.3.2 Aspects of Reference Interpretation . . . . . . . . . . . . . . . 43

2.4 Modifying Emergence . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

3 Reference Transformation . . . . . . . . . . . . . . . . . . . . . . . . 48

3.1 Concepts Underlying the Theory . . . . . . . . . . . . . . . . . . . . 49

3.1.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

viii

3.1.2 Different Forms Referencing the Referent . . . . . . . . . . . . 51

3.1.3 Keeping the Same Referent . . . . . . . . . . . . . . . . . . . 53

3.1.4 Emergent Re-formation . . . . . . . . . . . . . . . . . . . . . . 55

3.1.5 Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.1.6 Confidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

3.1.7 An Indirect Approach . . . . . . . . . . . . . . . . . . . . . . 65

3.1.8 Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

3.2 Reference Transformation . . . . . . . . . . . . . . . . . . . . . . . . 71

3.2.1 Different Forms Referencing the Referent . . . . . . . . . . . . 72

3.2.2 Deviation from, and Convergence to the Referent . . . . . . . 77

3.2.3 System Experience . . . . . . . . . . . . . . . . . . . . . . . . 80

3.2.4 Decomposition and Composition . . . . . . . . . . . . . . . . . 83

3.3 Emergence Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 87

3.3.1 Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

3.3.2 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

3.3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

4 Bridging Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

4.1 Surface as Metaphor . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

4.1.1 Folds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

4.1.2 New Folds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

4.2 Fitting the Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.2.1 Surfaces as constituents of Reference States . . . . . . . . . . 110

4.2.2 Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

4.2.3 Fold Populations . . . . . . . . . . . . . . . . . . . . . . . . . 121

4.2.4 Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

4.3 B-space to and from K-space . . . . . . . . . . . . . . . . . . . . . . 131

4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

5 Situations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

5.1 Discussing Situations . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

5.1.1 Situation Domains . . . . . . . . . . . . . . . . . . . . . . . . 141

5.1.2 Situation Complexity . . . . . . . . . . . . . . . . . . . . . . . 145

5.2 Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

5.2.1 Tomography . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

5.2.2 Wildlife . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

5.3 Levels of Situation Familiarity and Complexity . . . . . . . . . . . . . 158

5.3.1 Familiar Situations . . . . . . . . . . . . . . . . . . . . . . . . 160

5.3.2 Unfamiliar Situations; Familiar Folds . . . . . . . . . . . . . . 166

5.3.3 Partial Familiarity . . . . . . . . . . . . . . . . . . . . . . . . 173

ix

5.3.4 Sparse Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

5.3.5 Detailed Situations . . . . . . . . . . . . . . . . . . . . . . . . 185

5.4 Framework Implications for Situational Study . . . . . . . . . . . . . 188

5.4.1 Transformation Characteristics . . . . . . . . . . . . . . . . . 190

5.4.2 Implied Characteristics of Prior Calibration . . . . . . . . . . 192

5.4.3 Framework Modification . . . . . . . . . . . . . . . . . . . . . 194

6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

6.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

6.1.1 Flexible Intermediate Entities and Domain Independence . . . 198

6.1.2 Lateral Reference Transformation . . . . . . . . . . . . . . . . 199

6.2 Further Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

6.2.1 Requesting Further Evidence . . . . . . . . . . . . . . . . . . 202

6.2.2 Augmentation of Domain Knowledge and Experience . . . . . 203

x

Chapter 1

Introduction

A leopard in a jungle does not have to be seen, for its presence to be known[RSN03].

A trusted guide tells you that a leopard is due north of your position. This guide

might have recognized the warning calls of bird life, the movement of deer or received

information from sensors in the leopards territory. You might have heard the same

calls, or seen the same movement or even looked at the same equipment displays.

However, unlike your guide, you need these observations interpreted and presented

to you in a form understandable by you. This form could be sign language, natural

language, a picture of a leopard, or a combination thereof possibly accompanied

by pointing in a northerly direction. Alternatively, you could have a team of

trackers reporting their interpretations to you. They serve to filter the vast bulk of

information, so that you can decide which situation to look at in more detail.

The scenario above contains examples of different forms of indication. The

contextual domain, of discerning situations in a jungle, is used later in this thesis

as a case study.

1

CHAPTER 1. INTRODUCTION 2

1.1 Different Ways of Indicating the Same Idea

Both, original data and subsequent description, are ways of indicating situations in

the real world. The data is the original indicator; the descriptions are alternative

indicators to the same situation. An alternative indicator can have a different form

to the original data indication of a situation. Depending on the domain and the

recipient, the ease and appropriateness of indicators can vary. The term recipient is

used, as either a human or a machine could receive the final indication, possibly for

further interpretation. The work of the thesis shows how improve ease of indication,

given a descriptive form familiar to the recipient.

The thesis introduces a theory of transformation by emergence, and a framework

for transformation. The transformation is deemed to occur in parallel, resulting in

many small descriptions composing a greater situational description. The framework

can transform an available known form of reference to a more accessible, familiar

form. Such a reference transformation problem represents a simplified version of a

more general problem: that of improving indication using possibly multiple source

forms resulting in possibly multiple destination forms. The work on the specific

problem of reference transformation provides direction for further investigation into

the general problem of reference interpretation. The word ‘lateral’, in the term

“lateral reference transformation”, is used to note that reference transformation can

occur between representational forms of similar complexity.

Multimedia can be used to visualize data that is normally difficult to appreciate.

This notion is given new life in Chapter 3 where one form is transformed into another.

This includes the use of different media which complement each other. Radiation

from a star, or elements of a substance, can be broken into separate components, and

presented in a visual medium. The representation of the content of one medium in

another aids understanding. Communicated multimedia (or representations that can


be reconstructed as multimedia) can be used by remote recipients who do not have

easy access to the original environment. Doctors can manipulate probes in remote

patients based on imagery received. The remote environments can be reconstructed

or transmitted.

1.1.1 Indication

Communication

Indication is a form of communication. A traveler who is lost can ask for directions to

the intended destination; a local person can simply point in a direction. The pointing

conveys information in a way that the labels of “north” or “north west” might not,

if the traveler does not have a good sense of compass directions. The indications

which make most sense to the traveler, will be the more useful description. Ideally

the traveler ends up knowing where the destination truly is. Up until that stage, the

directions provided are considered by the traveler in relation to what is personally

known about the world.

One of the better illustrations, of the difference between indication and reality,

is the Zen parable of the finger pointing at the moon [Chu90]. A man reads a

philosophical text and has trouble understanding it. He takes it to meet a Zen

master known for wisdom. The master says that he will help the man understand

the philosophy, if the man will read it out to him. Surprised the man says “but

master, can you not read”. “No” says the master. “But how is it that you are so

wise without knowing how to read”. The master explains “if you wanted to know

where the moon is, I could help you by pointing at it. Following the direction

indicated by my finger, you would soon know for yourself where the moon is. But

my finger is not the moon, it merely indicates where it is. Words are not the truth,


but they help you find the truth”. This is the notion that a difference lies between

what is being pointed at (indicated) and that which points (indicates).

This thesis recognizes that data and descriptions are both just indicators of

entities or notions. Moreover, some indicators are more accessible by the observer

than others. The data gathered by a system might not be the best indicator for the

eventual observer. This thesis provides a framework for transforming one form of

indication to another form of indication.

Indication using existing knowledge

Knowledge possessed by a system needs to be convertible into a form of indication

appropriate for explanation. This will be easiest if the knowledge form is the same

as the form of indication. Such knowledge is assumed to have been bootstrapped or

otherwise deemed to already exist.

Knowledge, in this study, is correct indication of situation or notion. Specific

knowledge is indication of a specific situation or specific notion (situations are the

focus). Knowledge can exist at different levels of detail, such as classification and

description, through to deep appreciation or understanding.

Traditionally a description is an account of something in words. In this thesis,

a description is an account of a situation in a preferred representation. It is an

indication of adequate complexity for the task; classification is only description of

the roughest granularity.

Interpretation (verb) of data is the attempt to find a description which indicates

the same situation as indicated by the data. An interpretation (noun), is considered

to be a description generated after involvement of data with the system. A correct

interpretation (noun) is a description which indicates the same situation as indicated


by the data. An interpretation can be considered as emerging from the combination

of observation and experience, and represented using knowledge.

1.2 Emergence

When a quality or entity emerges, it comes from within a system. It is a phenomenon

that is intrinsic to the system; not something that is imposed on the system from

outside. The elements that are intrinsic to a system might interact with each

other. Emergence is modeled as the natural consequences of those interactions.

Traditionally, complexity of a system is seen to increase as side effect or benefit

of emergence. Emergence from self-organizing systems can be manifest in sensible

behavior or increased order. For some recent computational systems, meaning or

interpretation is the desired emergent quality.

Computational emergence is a slightly more contrived kind of emergence in

that the various elements can be brought together. In systems that experience

computational emergence, elements are free to interact in unanticipated ways, with

possibly surprising results. They differ from algorithmic and rule-based systems

that seek to anticipate or explicitly reason about variations of future situations.

The elements, that interact for emergent interpretation, can include observation

(e.g. sensor data) and experience (e.g. obtained directly or indirectly from users).

The Ant Colony metaphor

Douglas Hofstadter’s ant colony metaphor is a commonly used example of emergence

in nature ([BT00]). An ant colony is comprised, primarily, of many small units

known as ants, as well as the structures that compose their dwelling. Each ant

can only do simple tasks e.g. walk, carry, lay a pheromone trail, follow a trail,

etc. However, the colony is sophisticated enough to thoroughly explore and manage


its environment. More specialized ants can fight, look after young or the queen.

The most specialized - the queen - can produce more ants. However, generally, the

behavior of a single ant is quite simple. Collectively - the ant colony - is adept

at exploring its environment, farming resources, building complex structures, and

protecting them from predators and other dangers. Some ant colonies even establish

symbiotic relationships with plants and other animals.

Each ant colony grows and self-organizes from the interaction it has with its envi-

ronment and interactions between its components. The main emergent phenomenon

is self-organization, expressed in specialized ants being where the colony needs them,

when appropriate. These ants and others, the ant interactions, the synthesis and

self-organization compose the ant colony. This section describes the characteristics

and practical issues of emergent systems.

The metaphor of the ant-colony (or an aspect of the ant colony) is now modified

to suit the thesis. The pheromone trail is a simple indicator to ants where resources

are. It performs a similar function to the colored lines painted on hospital floors to

aid in the guidance of hospital visitors who need to find particular facilities. The

person painting a line might have been given a set of directions possibly involving

a map or a personal guide. The information (or indication) provided by the set of

directions is transformed into a colored line which is easier to use. A lines of paint

is the accepted destination form in the hospital; a pheromone trail is the accepted

destination form in an ant colony. In the thesis, the description indication is easier

for the recipient to follow than the original data indication.

If a pheromone trail is laid by another insect, an ant might not be able to detect

it let alone interpret its significance. Though an indicator is readily available it is

not readily appreciated or accessible to an ant. If the alien-pheromone trail could

be replaced by an ant-pheromone trail, access to the information or the indication


would be easier, for the ant. The ant (with its experience/general knowledge) could

interact with the detected pheromone trail (the data) to obtain an awareness (specific

knowledge/interpretation) of its particular situation.

Finding a way to transform an available indicator (the alien-pheromone trail)

into a more accessible indicator (an ant-pheromone trail) is the work of the thesis.

The two representational forms (types and positioning of pheromones) are of similar

complexity, but different accessibility. The transformation is lateral with respect to

complexity.

The transformation of indicator might well involve partial invocation of emer-

gence through interaction with the available indicator (interaction with the alien-

pheromone trail) in order to produce the alternative indicator (the ant-pheromone

trail). However, this can be seen as part of early emergence as there is interaction

with the alternative indicator to attain final awareness of the situation. The orig-

inally available indicator is also being used an element participating in interaction

to effect transformation. The resultant indicator can be used in further interaction,

possibly resulting in better awareness of the situation.

Other Analogies

The creation of footprints in the sand is an analogy of interaction. For this thesis

the interaction is that of data and knowledge. The footprints indicate the maker of

the footprints. The pressure applied by the human or animal making the footprint is

the analog of the data. The tendency of an observer’s mind to associate a footprint

with an animal. The observer uses its knowledge and memory to interpret the

footprints. The newly encountered prints are mapped or associated with the humans

and animals known to have made the prints found in the observer’s memory. The

feature data consists of footprints. The prints are laid out in a certain way. Each


print has a certain shape. The actual reality is that of a past event of a human or

animal making the footprints. The knowledge, used by the observer, is in the form

of prior experience of humans and animals making footprints. The interaction is

that of the footprints with the observer’s memories or experience. The construction

of an animal in the mind of the observer is an indicator to the past reality.

The interaction of entities taking part in emergence, and the interpretation of

the emergence result are described here with analogies in cell biology. Form is

analogous to the the three dimensional configuration of protein and lipid folds, which

themselves are re-expression of the needs of their environment. An insulin receptor

on a protein crossing a cell membrane, while interacting with other molecules will

cause membrane and protein re-conformation[Kan06]. Where “conformation refers

to the spatial relationship of every atom in a molecule”[RK03]. The re-conformation

of the membrane and proteins provide information to the entities within the cell. The

tertiary structure of a protein is composed of secondary structural units (“helices,

sheets, bends, turns, and loops”) that form substructures which correspond to

particular tasks. Information is transfered using the forms of the proteins and

membranes. The reality indicated by a description is analogous to phenotype or

behavior indicated by certain protein folds. The phenomena of the cell environment

transforming molecular structure and subsequent interpretation by the cell, is anal-

ogous to the framework described in this thesis. It is a difficult problem for human

researchers of proteins to interpret the significance of the unfamiliar conformations

of a protein. However, the cell is familiar with the protein conformations and can

act appropriately. Emergence through interaction is an important activity, but a

framework or process is needed to make sense of the results of the emergence.


Calibration of an emerged form

This thesis suggests a framework that aids in the interpretation of the references

that emerge. The emphasis is on aspects of the data that aren’t easily discerned,

and on transformation and emergence that isn’t easily entailed from the data. There

exist interactions that are non-obvious and indicators that are not explicit.

In philosophy there is a metaphor of shining a light on the darkness [Bor01].The

darkness is hidden reality, or in the case of computerized systems - the information

content hidden in the initial representation. In this thesis, the light is the process

of emergence, though not classical emergence that sees greater complexity system

arising from a less complex system. Early emergence, can be more lateral in

nature, where the level of complexity stays the same. However, a situation is better

communicated to a recipient, by changing the nature of indication to a situation.

The framework facilitates transformation to a destination form that more easily

communicates the significance of a situation to a recipient who has difficulty com-

prehending the original data. For the purposes of this thesis, a destination form is

assumed to be specified in advance; the choice of ideal destination form is outside

the thesis scope.

In disciplines such as physics and engineering, calibration is seen as a “process for

translating the signals produced by a measuring instrument (such as a telescope)

into something that is scientifically useful.”1 The purpose of calibration in this

thesis is to encourage association with descriptive elements (representable by the

destination form) that can better indicate the same situation as indicated by the

data. There is special interest in describing notions that the recipient finds hard to

express or discern in the original data form.

1http://imagine.gsfc.nasa.gov/docs/dictionary.html#calibration (Last viewed 17 June 2009)


1.3 Motivation and Scope

1.3.1 Making Sense of Non-text Media Content

The big picture involves the interaction of experience and specific situational data

to yield a specific description of the situation. Emergence is an appealing approach,

but it has its shortcomings. One problem is that the end result of the emergence

might not be in a useful form. Addressing this problem involved considerations of

knowledge representation, and observations on the indicative natures of description

and indication. There is no common baseline for equating representations or descrip-

tions of different media. A common approach is for a correlation to be agreed upon

between the sensory detections in one medium and the representation in another.

Some standards have a text description stream in parallel to the media stream.

The text accompanying a scene or a shot is taken as its ‘meaning’.

Context is not considered to vary from that of the author of the text. The same

assumption is made by this thesis with respect to calibration.

For example, a graph is often a simple way to convey information. It presents

the information in a visual medium, even if the data was collected by aural, haptic

and other sensors of other media. Even when the detection medium is also visual

(e.g optical telescopy), changing the visual form can be useful. The visuals of a

graph (e.g a graph of spectra) cause associations in the minds of the observers (e.g.

associating spectra with particular elements).

People from different countries describe situations using native languages which

differ from the languages used in other countries. A person proficient in one language

will not be able to comprehend an unfamiliar language. Some international signs,

such as those (for arrival and departure) at airports, omit words altogether.


It is unclear which details in the media are used by an observer to make an

assessment, such as a diagnosis. Weight analysis in neural networks can be used

to identify the salient inputs for classification problems. A goal in this thesis is

to generate descriptions. Descriptions can be used by calling systems to aide such

tasks as classification and diagnosis, however the service of description is neither

classification nor diagnosis.

There is motivation for research into non-verbal computing, where the users

are illiterate [Jai03b]. Systems cannot rely on the users being able to comprehend

natural language scripts. Without recipient ability to issue and access abstract

concepts, the concepts must be inferred. Experiential computing [Jai03a], [SSR03]

allows users to interact with the system environment, without having to build a

mental model of the environment. They seek a symbiosis formed from human and

machine, taking advantage of their respective strengths. Experiential computing,

while in its infancy, might in the future enable implicit relevance feedback. The

recipient’s interactions with the system can cause both emergence and verification

of semantics.

Current and proposed representational standards (e.g. MPEG-7, MPEG-21)

have a natural language description associated with a piece of multimedia. The

function of natural language is to act as a form of indication. There are other ways

of providing indication, such as contour lines on maps [Har99], images of animals or

pointing in a direction[SSFNY08].

It is desirable for machines to be able to use knowledge, about situations, without

depending on text. Descriptions can be expressed using things other than natural

language. For instance, an image of a leopard could be the meaningful description

for a collection of jungle warning sounds and footage of other animals scattering.

Sometimes it is necessary to know whether the leopard is sleeping or prowling.


1.3.2 Objectives

The principal objective is lateral reference transformation, to provide alternative

indicators to a specific situation, given available specific data and general domain

knowledge. The alternative indicators (descriptions) should indicate the same situ-

ation as the data, without changing the complexity of the situation being indicated.

It is this maintenance of complexity level that is implied by the term ‘lateral’. There

also needs to be confidence that the easy to comprehend description indicates the

same situation that the data was indicating.

An aspect of the transformation is the use of existing domain knowledge; is not

on the imparting of new general knowledge to a system store. A human spends years

acquiring information with brains more sophisticated than current machines. There

is no attempt to emulate experience acquisition. A store of experience is assumed.

Whether this store of experience was bootstrapped or incrementally grown is a

consideration outside the scope of this thesis.

Existing domain knowledge should be in the form desired for the eventual de-

scription of the situation. However, the approach used to achieve reference trans-

formation should be independent of the domain, and the representational form used

for the domain knowledge.

Transformation needs to make sure that the resultant description is in the de-

sired form. Intermediate representations should be sufficiently flexible such that

transformation, to the specified destination representational form, can be achieved.

As such, any intermediate representations, used during transformation, should also

be independent of the situational domain and representation form.

Some indicators in the data are replaced, in the description either by maps into

the domain knowledge or representations that are of the same form as the domain

knowledge. This follows the notion that description, even of something unfamiliar,


should be described in terms of something familiar. The domain knowledge form

is the form deemed to be of to provide indication that is more meaningful to the

recipient. Otherwise, the description would be as similarly unhelpful as the original

data.

Once a basic framework for lateral reference transformation is presented, it is

investigated in the context of one or more specific domains, to determine what can

be sensibly leveraged for improvements to the basic framework.

Assumptions

Sources of knowledge are assumed to both exist and have sufficient variety to be

able to describe a newly encountered situation. Time varying knowledge is outside

the scope.

Transformation is between two known forms. Each of the source and destination

forms can be non-trivially complex. Both will be known at the time of calibration.

Note that the kinds of forms are known, not the configurations specific to the

situational instances.

Data neighborhoods are assumed to be relevant to the current data. The corner

cases of empty data, empty knowledge and null situations are not considered. The

data has to be non-scripted and non-edited, so that there are no sudden context

changes. It follows that domains containing singularities (such as shot boundaries

in film) are not in scope.

Context is considered from two perspectives. From the perspective of the data,

it is the knowledge and the experience available for interpretation. This includes

entities and concepts that aren’t directly observable in the data. From the perspec-

tive of the description, context is whatever influences the emergence and subsequent

interpretation, without being explicitly indicated by the description.


Mechanisms of interaction are specific to application domains. Their existence

is assumed.

1.3.3 Structure of the Dissertation

The chapters are arranged in a way to explore how emergence can be used in trans-

formations between indications of similar complexity, but different form. The term

coined in this thesis - “lateral reference transformation” - regards early phenomena in

emergence where there can be transformation between forms of similarly complexity.

A form that is implicit to an expert, can be transformed into another form that is

explicit to its recipient.

The first chapter states the research motivations and objectives of this thesis. It

also introduces necessary concepts of indication and emergence.

The second chapter surveys the literature with regard to interpreting non-text

situational data. It begins with indication in non-text data. It continues with

the current thinking in the field of emergence. It demonstrates the difficulty of

the general problem of reference interpretation, and denotes a subproblem to be

for further investigated in the thesis. The chapter concludes by positioning the

specialized problem in the wider literature, and providing a road map to the rest of

the dissertation.

The third chapter is the heart of the thesis. It presents a framework and for-

malisms for addressing reference transformation from one known form of indication

(the data) to another known form of indication (the description).

The fourth chapter presents a possible approach for implementing the framework.

This includes a techniques for providing the principal framework component, inter-

preting the result of emergence, and estimating a confidence in the interpretation.


The fifth chapter undertakes case studies in the context of two domains: med-

ical tomography and wildlife observation. These case studies consider the initial

assumptions and implications of the transformation framework.

The sixth chapter summarizes the contributions made in the thesis, and provides

direction for further work in the area.

Chapter 2

Interpretation of Situational Data

In the previous chapter, it was seen that situations could be indicated by different

kinds of indicators. Indicators that are readily available, might not make sense to

some observers. Existing indicators if transformed might more easily be interpreted.

Relevant areas of the literature include those of meaning, interpretation, communi-

cation and experience. The chapter organizes discussion of the literature into four

sections.

The first section looks at indication in non-text data such as multimedia. It

begins with semantics and meaning; both in general and with respect to non-text

data. It covers classification and association of meaning, and ends in the area of

emergent semantics.

The second section looks at the forerunners of emergent semantics: classical and

computational emergence, with an eye toward alternative approaches for combining

emergence with meaning. Different techniques of emergent semantics are covered.

This leads to the problem of reference interpretation, a specialization of which is the

work of this thesis.

The third section considers the issues of the reference interpretation, with respect

to selecting aspects to address in the thesis.

16

CHAPTER 2. INTERPRETATION OF SITUATIONAL DATA 17

The fourth section considers the specialized problem, which is modification of

emergence at an early stage, by means of lateral reference transformation.

2.1 Indication in Non-Text Data

Indication often occurs in a primitive way computing, as means of location address

or identification. In the thesis, the term ‘reference’ is used in the linguistics sense;

an indicator of something in the real world - the ‘referent’ [Rei09].

Interpretation of complex data, can be of use to the recipient in varying granu-

larity, depending on context. Classification is a form of description that can seem

a rough summary or of coarse granularity. However, this can be a placeholder for

more detailed or sophisticated detail that has been previously agreed upon. Simple

classification can be sufficient, if the meaning associated with a classification is

appropriate for each member occupying that classification [MA01]. Though interest

often lies in the specifics or richness of a situation [DDN03].

The translation of natural languages (NL) [Nin09] is the closest analogy to the

transformation of indication in this thesis. However, the nature of the media involved

is quite different. The thesis intentionally considers forms of indication different from

natural language.2 Reference transformation, in the thesis, sees natural language

collectively as one medium. Natural language translation can take advantage of

sequential presentation of the source [Swi88], and the same medium for both source

and destination languages. Though grammars and morphology differ, there will be

commonality in concepts of grammar and form [Bit01]. Aspects of non-NL transla-

tion involve encountering and transiting between different media [DBLI07] and less

certainty with regard to what should be translated. Ostensibly natural language

translation deals with everything that is stated explicitly, though a good translator

2This excludes approaches where embedded text is extracted from other media.


or interpreter will be able to convey hidden meanings or creativity of expression

[Nin09]. The linguistic area of pragmatics3, where extra information lies in when

and how something is said, has parallels in emergence [Gri09], where what was

implicit is made explicit [GD97]. Though translation is not a specialization specific

to a particular discipline, the concepts of reference and referent (See section 2.1.1)

from linguistics are used. There is no comprehensive approach to address the issue

of multimedia semantics [SJ07].

There has been recent interest in combining the areas of multimedia, data mining

and knowledge discovery [PPT08][WN08]. Higher order mining [RSLC08] regards

mining without the luxury of primary data. Typical approaches to mining assume

primary data that is cleaned and prepared according to what the mining algorithm

requires. The intent is to provide better patterns for human comprehension. Asso-

ciation mining [CR06] is the task of finding correlations between items in a dataset,

and is user centric. Inferences are made from items or item-sets.

This relates to a study [SGJ01] involving users via user organization of image

collections into groups images, where the system then attempts to infer context and

meaning based on commonalities within each data group. [SGJ01] is more usually

mentioned in another recent area of research, that also looks at multimedia seman-

tics, called emergent semantics [RSN05]. Emergent semantics is a specialization of

computational emergence, which in turn was inspired by studies of emergence in

philosophy. It is covered in more detail in Section 2.2.2.

Associating meaning with what is observed, is a problem framed under two

well known headings - the “symbol binding problem” and the “semantic gap” (see

Section 2.1.1). The symbol binding problem is concerned with the meaning given to

or associated with symbols. It is dealt with in depth within the field of semiotics.

3This is similar to the notion of “reading between the lines”.


The semantic gap refers to difficulty in mapping low-level data to high-level meaning

[SWS+00][WZZ08][ZG02] .

To cross the semantic gap, an approach should result in transitions from ‘lower-

level feature data’ to ‘high-level concepts’. Current notions of emergence assume a

transition from low to high, where the quality being increased is complexity [Ult07].

If the resulting concept is indeed of higher complexity, then a transition from the

relatively lesser complex data is a transition from a low-level to a high-level. Though

this seems a side effect rather than a requirement.

2.1.1 Semantics and Meaning

The term semantics comes from the Greek Semantikos: significance or meaning.4

It’s typically used with natural language, but is being used more with regard to

the interpretation of multimedia data. Though natural language is not investigated

in this thesis, there are concepts from linguistics and semiotics that are relevant.

The notions of transformation and indication, in the thesis, are close in sense to

the linguistic notions of translation and reference. The symbol binding problem,

from semiotics, can be seen in the domains of non-text media. Classification can be

seen as a variant on symbol-binding, though there isn’t always an explicit symbol.

Sometimes there is an implied equivalence of meaning amongst all entities that

share a classification. Previously agreed meanings can be bound to classifications to

provide association between those meanings and categorized entities. The semantic

gap deals with a harder problem of automating the association of meaning between

media instances and meaning, in the absence of classification.

4Webster’s Online Dictionary http://www.websters-online-dictionary.org/se/semantics.html,Last accessed 24 June 2009


Linguistic indication and translation

Linguistics provides two pertinent terms: “referent” and “reference” [Rei09]. Both

terms are used in the thesis. Referents, with regard to multimedia, can be the

individual or collective significance of cues in multimedia. Different cues in different

media are able to hold the same significance [ZPRH07]. A reference is something

that indicates. It can also be considered a representation. Emotions can be referents;

observed facial expressions can be the references [Bre06].

A representation stands for some other entity, and is “more concrete, immediate

or accessible”[BL04] than the other entity or referent. Others means of indicating

a referent can be sought, by processing the original data indicators. For example,

data mining techniques hunt through data looking for information[WF05]. Data

can also be explored, looking for ideas by which to construct information. The

forms present in the data need not be govern the final form of indication of the

referent. Though knowledge representations are one way of capturing significance or

indication, another is behavior, such as robotic behavior [ABBS00], where modeling

is validated through appropriate behavior.

The function of a language is to provide the ability to make a reference to a

concept. A concept might be a real world situation, a notion, or a conveyance of

knowledge or experience. Atomic meaningful units of language are known as mor-

phemes5. The synthesis of description from smaller references is examined further

in Chapter 4. Most natural languages however have grammars that can be used to

aid in the translation from one natural language to another. In linguistics, these

languages are classified under the Chomsky Hierarchy[Cho02], where the grammars

are sequential.

5http://www.askoxford.com/concise oed/morpheme?view=uk, Last accessed 24 June 2009


Symbol-binding

A common medium in which meanings are bound to signs is a natural language

language. In the case of written languages, examples of symbols are glyphs and

ideograms. With a spoken language, meaning is associated with sounds or groups

of sounds. These sounds can also be associated with glyphs and ideograms. Cues

picked up from situational data can act like symbols if sufficiently familiar. Low-level

data, such as tempo and color, in a multimedia scene, can act as symbols [DDN03]

which reference emotions. With experience an observer can bind meaning to visual

observation. Visual semantics include notions of icon, index and symbol [DDN03].

An icon has similarity to referent properties, and are typically of the same medium.

An index is a representation that has an “inherent relationship” to the referent.

The meaning associated with a symbol is arbitrary, but agreed upon, and functions

much like an idiom in NL [KS96].

Meanings can be associated with entities in different media. Taking the linguistic

notions of referent and reference, the meanings (referents) can be abstract notions,

prior experience or even other indicators (references). Meaning can be in terms or

characteristics of whatever is easiest to comprehend [DGD08] Multimedia contains

multiple media, where entities in each medium or composite entities across several

media can separately be bound to concepts or each other [ZM08].

A multimedia scene is not reality; it is merely a reference to a referent in reality.

Similarly, the output from emergence is a reference, that can be used by the recipient

of the information. The linguistic terms ‘reference’ and ‘referent’ indicate existence

in the “model world” and the “real world” respectively. There is a danger in

confusing the two [Min88]. The referenced meaning is embedded in the experience.

This is similar to attribute binding using Dublin Core metadata[Hil05][Dub08],

where the standard attribute name is associated with the commonly understood


semantic. There is also work on matching representations, to establish semantic

mappings between representations [Doa02]. This is similar to the equivalence of

indicators in different forms. The same information or indication can be present in

different forms. Implicit information in one form can be more explicitly presented

in another. Gero defined emergence as the process of making explicit what was once

implicit [GD97]. He used genetic algorithm genotypes (evolving strings) to represent

knowledge. The strings can contain units. Interaction is enabled by operators that

act on them. The genotypes evolve over several generations, with the successful

genes selected to generate the next generation. Fitness functions determine success,

and influence how gene strings evolve [GD97].

Context is taken from the domain, the data instances, or the recipient. A

recipient’s context is mainly taken from their interaction history. Their personal

history and current mental state are harder to measure. The recipient’s role during

context gathering can be active (direct manipulation) [SGJ01] or passive (observa-

tion) [Ker02]. These activities provide binding for data features.

Metadata (data about data) can be used as an alternative semantic description of

multimedia content. MPEG-4 is currently a video standard that deals with objects

and motions [DDN03]. MPEG-7 augments video by adding a description stream,

which is associated with the multimedia stream by using temporal operators [TD08].

The description resides alongside the data, and can be generated from ontologies

[BDBS08] or non-text sources[SGS08]. However, it is difficult in advance to provide

metadata for every possible future interpretation of an event. MPEG-21is a proposed

standard for context and usage [TAC+08][Per07]. Though currently set once and

fixed, the MPEG semantic stream can be replaced or augmented by acceptable

contextual interpretation. Thereby it could be of more use for a given user’s context.


If parallel data is employed as a knowledge source (description), as well as a

data source (multimedia), there is an assumption of fixed context for all recipients

of the content. Ideally a recipient could access an interpretation with a basis in

the context of the recipient’s information need. Without interacting with the user,

the recipient’s context is difficult to obtain. Less ideally, for the recipient’s possible

edification, one or more experts’ context could be used as a proxy for that of the

recipient.

Unfortunately, an expert will often not have the time to diagnose massive stores

of data, or always be on hand to craft a description for each portion of multimedia

data. Autonomic systems can help to filter the data through classification, or more

thoroughly process data through diagnosis.

Several multimedia systems attempt semi-automatic annotation, but depend on

a training corpus of labeled data[Hun08]. Some approaches derive natural language

from predicates associated with modeled concepts [KTF02]. Image regions can

be separately annotated with labels[SGS08] or descriptions. [SD08] presents an

argument against formal annotation, based on two assumptions:

Firstly, that the meaning of a document is given by the change it provokes

in the context of the activity in which it is read; secondly, that these

activities can be configured as games, and that what is usually called a

query is but a type of move in these games. Simone Santini[SD08]

[EPKM+07] offers a media interpretation framework based on low level semantic

extraction techniques, using abduction to determine causality. The outputs are

descriptions of multimedia documents. The measures used are standard in infor-

mation retrieval - recall and precision[SM83]. In abductive reasoning, a system

tries to obtain what would be required to believe a result, when given a knowledge

source. Knowledge representations can be used to encode and store the meanings


determined. The interpretation is embodied in low level annotation data combin-

ing with contextual and conceptual knowledge to yield “enriched annotation data.

Related techniques exist in video data mining [MP05] and low-level sensor fusion

which uses pattern matching as inference against an already established knowledge

base [NLF07].

Classification

Classification or categorization[MA01] is a technique which enables each member,

of a group of entities, to be considered equal in significance to each other member

of the group. A classification might not have significance or interpretation beyond

the implied sameness between group members. Alternatively, each member can be

considered as having a previously agreed upon meaning or significance, as associated

with a particular category, which might or might not be associated with a form of

identification. A category can be considered a description of very large or coarse

granularity. It does provide a different form of indication to a referent otherwise

indicated by the data. Constituent indicators, within the data, may separately or

collectively indicate several individual referents or fewer composite referents.

Though sometimes a classification or a label is sufficient, there are times when a

more detailed explanation in an accessible form is desirable [Nin09]. A classification

implies an agreed upon meaning understood by both the classifier and the user of

the classification. In the sense that classification is a simpler form of description,

and that current interpretive systems are more constrained in their scope, the

thesis conceptually subsumes both. It offers an approach to interpretation that

is independent of domain.

User observation and user interaction lend themselves to highly localized views of

the situation. The thesis takes an open view, and allows for multiple interpretations.


If recipients want to contribute they could, but user participation is not mandatory.

The user could contribute temporary knowledge or additional data to help interpret

a specific situation. Though the knowledge provided is different to the simple

classification that is usually found in techniques such as relevance feedback[HL08].

Description can vary in sophistication from classification, through simple associa-

tions [BDFF+08] to details specific to a situation. In automatic concept annotation,

concept correlation can be used to provide information beyond single concept clas-

sification and annotation [QHR+07]. Generation of a description can be seen as

a composition from multiple related classifications for the same data. However, it

does not provide description of specifics of the data situation, more so what the data

situation has in common with other data situations with the same classification.

The acceptability, of a level of description granularity, will be dependent on the

situational domain.

Semantic Gap

The “semantic gap” [SWS+00][WZZ08][ZG02] is a metaphor for a qualitative differ-

ence between low-level features and high level concepts. It is a general problem in

the areas of machine learning and reasoning. The sophisticated notions by human

observers of a medium, and simple classification or object recognition, differ in

complexity. Though humans might determine a classification, they have internal

models or appreciations of the situation indicated by the data. The ‘Semantic Gap’

is industry jargon for the gap between sensory information and the complex model

in a human’s mind. Smeulders[SWS+00] defined the semantic gap as “the lack of

coincidence between the information that one can extract from the visual data and

the interpretation that the same data have for a user in a given situation” . An

issue arises when the semantic gap needs to be bridged for situations, when a user


is not available. The same sensory information provides some of the units, which

participate in computational emergence. The semantic structures, which are formed,

are the system’s complex model.

Some algorithmic approaches seek to narrow the gap by attempting semantic

detection through concept fusion[WN08][SWS05]. Pattern recognition is performed

of semantic concepts at shot level. Fusing occurs either at feature or semantic

levels. The data and the description are both often better indicators than simple

classifications and labels. There are often indicators in the data that are not well

captured in an automated description [DBLI07].

The gap between different indicators is not necessarily one of qualitative differ-

ence in information held. However, there can be a perceived qualitative difference in

the description with regard to the ability of a third party recipient to comprehend

the indication. Especially, when the recipient has not the experience of the initial

observers. Both forms of indication can be of the same complexity, but one can be

more useful.

Transformation between different forms of indication are analogous to attempts

to bridge the semantic gap, between low-level data and high level concepts. Emer-

gent semantics is an attempt to attain the bridging of the semantic gap using the

benefits of emergence.

2.1.2 Experience

Experience is used to help interpret observation or indication. Experience is defined

as “practical contact with and observation of facts or events”6. In neuroscience

repeated experience can lead to association [LG09] by reinforcement. Learning can

be associative where experience is based on event or stimuli pairs [Rei01]. System

6Concise Oxford Dictionary, http://www.askoxford.com/concise oed/experience?view=uk,Last accessed 24 June 2009


experience is used as a proxy of human domain expertise or experience. It simulates

prior practical knowledge with familiar situations, in an attempt to interpret new

possibly unfamiliar situations. Measuring the accuracy of world modeling can be an

indirect inference from the phenotype (behavior) of the system or robot population

instead of an assessment of the genotype (representation). Driverless vehicles are

assessed by how far they travel, and how well they interact with obstacles [UW08].

If the behavior is sensible, the representation is assumed to be fit. Learning, in this

context, is a change in behavior based on experience [Rei01].

Information from ontology can be used in decision making, or in suggesting other

units for interaction. Kuipers’ terrain ontology [Kui00] assists interpretation of robot

perception and navigation by providing co-occurrence information with respect to

perception hypotheses of the robot. The twin sources of error, of sensor and motor,

correspond to errors in situational data input and ‘errors’ in indication output.

The ontology includes information for sensory events, causality and topography.

Multiple representational instances exist. An ontology like this works with partial

sensor information from the robot, consultation with the ontology, and verification

in subsequent sensor readings. This aids decision making. The amount of forward

planning depends on applicability and certainty of aspects of the model to the robots

situation. Information about its environment enables a robot to orient itself in

the neighborhood without needing to travel. The better the model of the terrain

situation the less actual work required to comprehend the situation.

2.1.3 Communication

Communication is broadly concerned with representing, imparting, conveying and

displaying knowledge. Saussure’s conduit metaphor [Sau83], in semiotics, has the

notions in a person’s mind transformed into language elements. Speech is the conduit


through which the language elements travel. They are transformed back into notions

in the listener’s head. [Dav03] sees the conduit metaphor as too optimistic in

communication as effortless and relegating “communications breakdown” as rare.

The other main approach to semiotics is that of Peirce’s interpretive semiotics,

which in turn has influenced Eco’s semiotic theory of signs[Vio01] where cognition

is seen as an inferential process using signs. [AF08] investigates communication

models, beginning with Shannon’s Communication Model, and ties them together

based on flow and ontology. Visualization of data and information are means of

communication by usage of different forms for easier comprehension[WZ08][CKB08].

The communicated signs and references can be made more comprehensible through

the use of common knowledge.

An expert viewing the data might possess innate knowledge or nativism[Sam01].

This is the notion that some knowledge is built-in to the observer. So much so that

even an expert observer might not be able to provide an explanation or description.

Tacit knowledge is known without a readily communicable explanation, and has

aspects that are functional, phenomenal and semantic [Pol97]. These combine to an

ontological aspect which indicates what kind of knowledge the tacit knowledge is.

It is seen in day-to-day phenomena, such as muscle-memory or kinesthetic learning

which enables a person to walk or to throw a ball. This kind of knowledge can

be effectively communicated [LB07], though by means quite different to natural

language.


Ontology and Knowledge Management

Knowledge representation, for communication, can include ontology, metadata and

genetic algorithm strings. The use of templates and grammars, which can commu-

nicate semantics in terms of the media, aren’t emergent techniques as they have

predefined semantic structures with anticipated multimedia content.

Information needs, for a recipient, can be task dependent, with the task itself

evolving and not known beforehand. In such situations, the semantics and structure

also evolve, as the recipient interacts with the content, based on an abstract notion

of the information required for the task. That is, recipients can interpret multimedia

content, in context, at the time of information need.

Classically, ontology is the study of being7. The computer industry uses the term

to refer to taxonomies, entity/fact collections [TMKW07][SKW07], repositories of

properties and relations between objects, and semantic networks[RBVdS07]. Some

ontology is used for reference, with multimedia objects or direct sensory inputs being

used to index the ontology [Hoo01][Kui00]. Other ontology attempts to capture how

humans communicate their own cognitive structures.

The Semantic Web attempts, to use ontology, to access the semantics implicit in

human communication [Mae02]. Semantic networks consist of a skeleton of low-level

data, which can be augmented by adding semantic annotation nodes [Nac02]. The

low-level data consists of the multimedia or ground truths, which can act as units in

an emergent system. The annotation nodes can contain the results of emergence, and

are not permanent. This has the advantage of providing metadata like properties,

which can also be changed for different contexts.

7Oxford English Dictionary www.oed.com, Last accessed 21 June 2009


Ontologies [KHL+07] can include sets of concepts and interrelations, along with

concept role relations and attributes. Ontology is a “specification of a conceptual-

ization” [Gru93]. Entities and interrelations that are presumed to exist in particular

domains. Attempts have been made to extend linguistic ontologies to multimedia

[BDBT06] [PPT08] [SK07]. Sometimes, taxonomies or linguistic terms are suffi-

cient for content that need simply be categorized. Some other ontologies combine

linguistic terms with media such as pictures, audio and video clips. Alternatively,

multimedia data can be integrated with existing symbolic data in organizational

frameworks [GS00]. Concept similarity across different multimedia ontologies has

been researched as part concept detection [KS07]. Large multimedia ontologies are

being constructed for information retrieval research [Ove09][YCKH07]. Notions of

concept similarity have been based on visual similarity, co-occurrence and hierarchi-

cal taxonomies. An aim of multimedia information retrieval is to be a step toward

knowledge discovery in all forms[LSD+06].

2.2 Emergence

2.2.1 Classical and Computational Emergence

Classical Emergence

Emergence is the phenomenon, of complex structures arising from interactions be-

tween simple elements. Though, it might seem that nothing particularly happens

at the scope of two elements interacting, the collective effect of many elements

interacting can realize properties not observable at the element level [Qui06]. While

emergence is a relatively new concept in multimedia, it has been used in fields such

as biology, physics and economics, as well as having a rich philosophical history.

The term emergence was coined in 1875 by George Henry Lewes [McL01]. British


emergentism required non-linearity for emergence. Though a later notion of emer-

gence based on supervenience did not require non-linearity [McL01]. Properties or

features appear that were not previously observed as functional characteristics of the

units. Though constraints on a system can influence the formation of the emergent

structure, they do not directly describe it. “Emergence is generally understood to

be a process that leads to the appearance of structure not directly described by the

defining constraints and instantaneous forces that control a system” [Cru94].

Computational Emergence

Computational emergence builds on the discussions of emergence in philosophy and

on observations of the natural world.

The principal benefit of emergence is dealing with unanticipated situations. Units

in unanticipated configurations or situations will still interact with each other in

simple ways. Emergent systems, ideally, take care of themselves, without needing

intervention or anticipation on the part of the system architect (Staab 2002). How-

ever, the main advantage of emergent semantics is also its greatest flaw. As well

as dealing with unanticipated situations, it can also produce unanticipated results.

They might be useful, trivial or useless, or in the worst case - misleading. However,

the scope of output can be constrained by constraining the inputs and the ground

truths. Sometimes, a structure is better understood, if alternate possible forms can

be appreciated. Multiple (possibly competing) interpretations can be presented to

the recipient even if the schedule of presentation is based on a single interpretation

from each instance.

The foundation for computational emergence is found in the Constrained Gen-

erating Procedures (CGP) of John Holland [Hol00]. Initially there are only simple

units and mechanisms, which are transition functions. These mechanisms interact to


form complex mechanisms, which in turn interact to form very complex mechanisms.

This interaction results in self-organization through synthesis. Relating the concept

of CGP to multimedia, the simple units are sensory data, extracted features or even

multimedia objects.

There are different qualitative levels observed before and after emergence with

respect to structure, behavior, organization and timescale [SPT06].

2.2.2 Emergent Semantics

Interacting units can come from sources such as knowledge bases and available

situational data. Semantic emergence occurs when meaningful behavior (phenotype)

or sophisticated semantic representation (genotype) arises from the interaction of

many of these units. This includes user interaction, the influence of context, and

relationships between media. Emergent semantics, in the context of the thesis,

regards the emergence of significance of situational references.

Emergent properties often exist at a macro level, beyond analysis of the compo-

nents at the micro level [HSH+08]. The macro system is made up of the micro-system

interactions. Widening the field of view, to take in the interaction or blending[AW05]

of many units or sources, sees synthesis of complex semantic units, and eventually

self-organization of the unit population into semantic structures. Techniques of

emergent semantics include direct user involvement [SGJ01], observation [GSF02]

and self-organization [ACMH03].

Involving the user

The user can actively communicate context to the system, through direct interaction

[SGJ01][SNN+08][SBC07]. [SGJ01] ask their users to organize images in a database,

so that the system will be able to infer the binding significance of each organized


group. They use the example of a portrait. If the portrait is in a cluster of paintings,

then the semantic is ‘painting’. If it is in a cluster of people, the semantic is ‘people’

or ‘faces’. The same image can be a reference to different referents, which can

be intangible ideas as well as tangible objects. CollageMachine [Ker02] is a web

browsing tool which tries to predict user browsing intentions. The system tries to

predict possible lines of user inquiry, and selects multimedia components of those, to

display. Reorganization of those components, by the user, is used by the system to

adjust its model. Though, sometimes the user will be interested in many, not always

similar, items. [KKS+07] The emergence of significance is subsequently cognitive on

the part of the user.

Observation

The context of a multimedia instance is taken from past and future subjects of user

attention. Even if the authors of the media had a specific interpretation in mind,

a user’s interpretation can be different to the authors’ intentions. The entire path

taken or group formed, by a user, provides an interpretation for an individual node.

The entire path provides the context for visited component. Emergence is dependent

on what the user thinks the data is, though the user does not need to know how they

draw conclusions from observing the data. The emergence of semantics can be made

through observation of human and machine agent interaction [Sta02]. Context, at

each point in the user’s path, can be supplied by their navigation [GSF02]. For

instance, the World Wide Web is considered a directed graph (nodes: web pages,

edges: links). Adjacent nodes are considered likely to have similar semantics, though

attempts are made to detect points of interest change, as those points will mark the

boundaries of browsing path sections, each of which will contain related items of


interest. The meaning, of web pages (and multimedia instances in general), emerge

through use and observation.

Agreement

Local schemas can be agreed upon by accessing publicly known schemas or ontolo-

gies. With these in place, relationships can be introduced between other information

systems that only partially overlap. Region synthesis leads to self-organization of

semantic agreement[ACMH03]. Part of the approach is meant to address adaptabil-

ity to unfamiliar schemata. Agreement can be used to capture relationships later.

Emergence occurs when pairs of nodes in decentralized P2P systems attempt to

form global semantic agreements by mapping their respective schemas. Multimedia

semantics can use event aware models and core ontology[Hun03], to provide a

common understanding of base entities and relationships.

2.2.3 Complexity

Emergent computation is based on the idea that appropriate complex

structures might arise purely from the physics of the task environment,

rather than from an architect’s elaborate considerations.

Steffen Staab[Sta02]

There are two notions of complexity which get confused with each other: com-

putational complexity and complexity of expression. Computational complexity is

an estimation of the processing time required for a computer to perform very large

operations and algorithms. Complexity of expression relates to the sophistication,

subtlety and detail by which a referent is represented or described. The latter is

more relevant to this thesis. Though it is the complexity of situational referent that

is more pertinent than the complexity of reference.


Complexity is a quality that can change as a consequence of emergence. The

subtlety and detail of a situation indicated by the data, should be mirrored in the

subtlety and detail of the situation indicated by the description. Other possible

qualities affected by emergence include order [MS04] and clarity. Both complete

order (regularity) and complete chaos (randomness) are very simple. Complexity is

said to occur between the two, at a place known as “the edge of chaos” [Lan90].

Emergence results in complex systems, forming spontaneously from the interactions

of many simple units. In nature emergence is typically expressed in self-assembly,

such as (micro-level) crystal formation and (macro-level) weather systems. These

systems form naturally without centralized control. Similarly, emergence is useful

in computer systems, when centralized control is impractical. The resources needed,

in these systems, are primarily simple building blocks capable of interacting with

each other and their environment [Hol00]. Not all possible complex systems, that

may form, are interesting. The interest lies in systems that form potentially useful

semantic structures. Desirable environments are those where the emergence is likely

to result in sophisticated semantic representation or expression [WG02][Cru93].

Unless the meaning of the situation is simple and non-specific, classification will

not be a sufficient level of description.

A complex situational referent, can hold many points of interest. Context will

determine which aspects are important, except in the case of abnormal events which

can be of interest merely for being abnormal.

2.2.4 Interaction

The defining characteristic of useful emergent systems is that useful structures arise

from the interactions of simple units. Non-linear interaction is the notion that units

in the system interact to form a combined entity, which has properties that neither


unit has separately. The interaction is significant; examining the units in isolation

will not completely explain the properties of the whole. For two or more units, to

interact, some mechanism must exist which enables them to interact. The mere

presence, of two units, doesn’t mean that they are able to interact.

If the amount of data is too small, there might not be enough interaction to

create a meaningful structure. A higher amount of data increases the number of

units and types available. Unit pairings increase exponentially with increasing units.

A system where all units try to interact with all other units, can stress the processing

power of the system. A system, without all possible interactions, might miss the

salient interactions. It is uncertain whether increasing data richness leads to finer

granularity of semantic structure, or lesser ability to converge to a stable structure.

Before being able to reap the benefits of units interacting, units are needed.

These units are supplied from (or implied by) the data. Explicit selection, of units,

by a central controller is not part of an emergent process. Emergence involves

implicit selection of the right units to interact. The environment should make it

likely for salient units to interact. Possibly all units interact, with the results of

salient unit interaction being more easily interpreted. Genetic algorithms can be

used to lessen the space of possible interactions between units[GD97], possibly by

isolating the more important elements. Knowledge and context are acquired across

generations.

Interaction of all data bypasses the need to determine salience of data prior

to interaction. Though this may require participation of all units, placing a high

computational load on the system. Ideally, salient features (or patterns of features)

should naturally select themselves during emergence, or results from emergence,

from the salient features, might affect the results more. Simply having all the


necessary sensory (and other) information present will not necessarily result in

interaction occurring.

Choosing a data portion, to process first, is similar to the visual processing of

humans, where the choice of initial foci (seed data) provide the basis of visual inter-

pretation, with subsequent perception modifying the original hypothesis [AL03]. To

mimic this in emergence, salient units can interact first, with gradual incorporation

other units later. If salient units are known of before emergence or determined

a priori, object recognition could be used to anticipate the existence of particular

objects, such as a ball in a park. In such a case, the best match for the ball will

be found. This matching could drive the tracking of a desired event or inference

of other objects related to the assumed object [YXL+03]. However, in different

contexts, different units will be the salient units. It will not always be clear whether

it is better to seek cues, or prepare to respond to encountered cues. The context

should change which units are more likely to interact, or the significance of their

interaction.

Existing studies have mechanisms based on particular data types their systems

work with. Ideally, emergence would be able to support the interaction of any

entities, or at least any entities that can influence expert interpretation regarding

the referents of a situation. A simplification of this is to be able to have interactions

between all data reference types supplied as feature inputs to the system. Part of

the paradigm of emergence is the notion that implicit cues take part in interaction,

rather than just the cues that an expert or system architect might be explicitly aware

of. If all units need to be able to interact with all other units, there would need to

be O(n2) interaction mechanisms, when n is the number of data types in the feature

data. This issue is addressed later in the thesis. Along with the aspect of interaction,


emergent systems have aspects of self-organization. These in turn typically do not

have requirements for the form of what emerges from the interaction.

2.2.5 Self-organization

Self-organization is an oft-occurring aspect of emergence[GYWA05], and it is not

always helpful. It helps in that it lessens the burden of anticipation; it is problematic

in that the results aren’t always useful or comprehensible.

Self-organization typically involves populations of units, which appear to de-

termine their own collective form and processes. “Self-assembly is the autonomous

organization of components into patterns or structures without human intervention”

[WG02].

The self-organization in Artificial Life attempts to mimic biological systems

[Gri09][Rey87] by capturing “an abstract model of evolution” [Wal92]. Organisms

have genes, which specify simple attributes or behaviors. Populations of organisms

interact to produce complex systems. Individual units can feel environmental pres-

sure of feeding and predators [Gri09]. Emergent phenomena are experienced by the

population as a whole. Boids [Rey87] synthesizes flocking behavior in a population

of simple units. Each unit in the flock follows simple laws, knowing only how to

interact with its closest neighbors or immediate environment. The knowledge of

forming a flock isn’t stored in any unit in the flock. Unanticipated obstacles are

avoided by the whole flock, which reforms if split up.

Autonomic systems are computer systems that self-manage [HM08]. Their aim is

to lessen the need for human involvement. They are based on biological autonomic

systems that deal with unconscious reflexes. Part of the aim of emergence is to

enable machines to deal with what might be unconsciously interpreted by a human.


Humans might later use the work of the autonomic systems, similarly to the use of

context-aware systems.

Robot perception can use information fusion; the autonomous version of con-

text/semantic synthesis [TvdM01]. [AAYA08] image sees media understanding as a

problem and also sees fusing data from multiple areas of knowledge and expertise

to be necessary for creating an awareness of an environment.There is a notion of

representational limits [AAYA08] when converting sensed data into a useful form.

Low-level approaches eschew gathering data that cannot be converted into efficient

semantic interpretation. However, this pre-supposes that it is known what task is

relevant and which data is significant.

There is analogy, to emergence, in the field of neural networks. They do not pre-

suppose significant data, though link weights can be later analyzed to determine im-

portant inputs. They simulate neuronal firing, similar to that accompanying muscle-

memory knowledge. And they classify without providing explanation. Interaction

in emergence is analogous to neuronal activity. The classification arising from the

neuronal activity is analogous to the resultant entities arising from the interaction

that is part of emergence. It is seen as a technology that could be employed as

part of emergence, as opposed to being a different paradigm. Self-Organizing Maps

have been used in computational emergence [Ult07]. There is also data mining

research into multimedia using Self Organizing Maps (SOM). Though neither field

is concerned with semantics [PKK03][SZ00]. For emergent semantics, there is a

departure from the analogy, in that the significance of the data is sought more so

than seeking the particular input references that help identify the significance.

Since emergence is not something overseen, there cannot be certainty that the

system’s complex model is the same as the human’s complex model. An ant colony

is not controlled, though it is considered successful. If the ant colony self-organizes


in a different way, that structure would, most likely, also be considered success-

ful. There may be many acceptable, emergent semantic structures. The semantic

emergence is appropriate, to either the recipient or task, if it can be interpreted

usefully. The emergence is evaluated either through direct communication of the

semantic structure, or though system behavior. Some artificial life approaches seek

to influence or control emergence by indirect interaction, and by understanding and

exploiting emergence [QQL06].

2.3 Reference Interpretation

Much of the current research, into the semantics and meaning of data, is centered on

context. The premise is that meaning is not obtainable, unless context has first been

determined [SGJ01]. In this section context is discussed, and then other aspects of

reference interpretation.

2.3.1 Context

The word ‘context’ is derived from the Latin contexere8 (to weave together, connect).

Context-aware systems interweave experience or interest with environment to yield

meaning[BCQ+07]. In data bases and projects such as the Semantic Web there is the

notion of “data tailoring” [BCQ+07] [BQR08] via the composition of views overlying

the data schema. This thesis deals with less defined data, such as multimedia.

This notion is extended to forms of indication, where a view can be considered a

modification of the form or medium of representation or indication. “Indication

tailoring” is the corresponding notion, as the term ‘data’ is used to identify the

original means of indication.

8Online Etymology Dictionary, http://www.etymonline.com/index.php?term=context, Lastaccessed 24 June 2009


Context influences emergent structures. Useful mechanisms, enable different

interactions to occur depending on the context. These mechanisms either implicitly

select, from the data, the salient units for each situation, or cause interaction of

those units to have a greater effect.

To evaluate emergence, there must be either observation of system behavior

(phenotype) or measurement or inference of knowledge representation (genotype).

The system representation must be translatable to terms a recipient can understand,

or to an intermediate representation that can be interacted with. Typically, for this

to be possible, the domain needs to be well known. Unanticipated events might not

be translated well. For the purposes of the thesis, data sources will be considered

appropriate only if they are unedited, and preferably continuous, with no avoidable

context switches. This is to avoid a distorting effect, known in film theory as the

Kuleshov effect[Kat91][KS06]. Reordering shots in a scene, affects interpretation.

The unanticipated must be communicated in terms of the familiar. Emergence

must be in terms of the system being interpreted. Otherwise there is a risk of infinite

regression [Cru93]. The environment, context and recipient should be included as

part of the system. Semantic structures, which contain the result of emergence, need

to be part of the system.

Context either determines which of the many interpretations are appropriate, or

constrains the interpretation formation. Context is taken mainly from the recipient,

the immediate environment or from the application domain.

Spatial and temporal positioning of features can also provide context, depending

on the domain. The significance of specialized information, such as geographical

position or time point, are part of application domains such as fire-fighting or

astronomy.


Context supplies the system with constraints on relationships between entities.

It can also affect the granularity and form of semantic output: classification, labeled

multimedia objects, metadata, semantic networks, natural language description, or

system behavior. Different people want to know different things.

Context can come from sources other than recipients. Multiple media can be

associated with data events to help in disambiguating semantics [NK97]. The

domain, in schemata agreement, is partly defined by the parties involved. Some

current approaches involve the user as a participant in emergence. The semantics

emerge through interaction of the user’s own context with multimedia components

[SJ99], [Ker02]. The user decides which entities interact, either actively or passively.

Some context-based systems attempt to assist the recipient by processing some of

the recipient’s context [FRLK08]. The recipient’s final interpretation of the situation

is composed from their own direct observation and also context-based systems.

Context helps to deal with the problem of subjectivity, which occurs when

there are multiple interpretations of a multimedia instance. World knowledge and

context help to select one interpretation from the many. Ideally, semantic structures

are formed that can be understood by third parties who do not have access to

the multimedia instance. This is not the same as relevance to the recipient. A

system might want to determine what is of interest to one recipient, and have that

understood by another.

Templates can be used to search for units suggested by the ontology. Well-known

video structures can be used to locate salient units within video sequences [Rus00]

[VD01].

With direct manipulation and recipient observation, synthesis and organization

come in the form of users putting things together. The current literature either


provides emergence models for particular contexts, or provides context by involving

the user as a participant in emergence.

2.3.2 Aspects of Reference Interpretation

Perception

In order to model what humans can observe, it is often helpful to model human

vision. In computer vision, grouping algorithms (based on human vision) are used

to form higher-level structures from units within an image [ES03]. An algorithm

can be an emergent technique, if it can adapt dynamically to context. Perception is

seen in terms of relationships between sensation and situational model. The sensory

modeling within the human mind is based on interpretation of electric signals from

the sensory organs [KJ08].

Composition of interpretation from multiple sources

The information from multiple perceived sources can be synthesized or composed

into a recognizable whole. Synthesis is essentially the interaction mechanism seen

at another level or from a different perspective. A benefit of emergence is that the

system designer is freed from having to anticipate everything. Synthesis involves

bottom-up emergence, which results in a composite structure. It lessens the need,

for high-level control that tries to anticipate all possible future scenarios and combi-

nations of information or indication.However, the unanticipated interaction of simple

units might result in an unanticipated and complex structure, which is difficult to

interpret. Notions of scale are with respect to domain and complexity of situations.

Complexity refers indirectly to the sophistication of referent available, and the many

ways in which data and knowledge can interact.


Situations

The significance of situations can be collectively indicated from various sources in

the situational data. Situations have attempted to be formalized as histories of

events[LPR98]. They can also be conceptualized as the environment of the recipient

of system descriptions [PRB+07]. Situations can be seen and described in terms of

roles and activities [Cro05] .

Significant situational data can be sparse within the available data. Explicitly

identifying units for interaction can be a practical non-emergent step, though it runs

the risk of ignoring implicit or subliminal cues. Units can be feature patterns rather

than individual features [FGLH03]. Some techniques attempt isolation and selection

of salient data, such as events [WJ07].

Situations can have multiple indicators. Situation specific indicators interacting

with experience can give rise to emergence of meaning of the specific situation. If

sufficient experience does not exist, interpretation can be less meaningful.

Reference Improvement

The problem is manifest when conventionally available means of indication is not

accessible by the target audience for whom the available. For instance, text is

not accessible by an audience that is illiterate[Jai03b]. In some cases, reference

transformation is appropriate as the kind of reference improvement. For example, a

machine interpretation of a visual environment can be transformed to haptic signals

and communicated to a visually impaired recipient [PRB+07].

Reference improvement is a variant of reference interpretation. It can exist by

augmenting existing data references. For example maps can be annotated with

labels to show locations, or overlain with contour lines to show points of equal

elevation[Har99]. Augmentation might require preservation of the original data


references, which will be used as a base for the improved indication of a situation.

The improvement should outweigh the error incurred during the act of improvement.

2.4 Modifying Emergence

However, instead of starting the process of improvement, after emergence has already

started on what is initially available, more flexibility can be sought by altering the

base of emergence to elements that are more intuitive to the recipients of the emerged

interpretation. It can either be seen as early emergence or pre-processing.

The problem is simplified by providing one possible alternative, a representation

form known to be more accessible to the user/recipient. There is no existing

specialized literature for the area of altering the basis of emergence. This steps

back from the problem of situational reference interpretation.

The more specialized problem is the modification of emergence to enable better

interpretation. The problem is limited to indication improvement by indication

transformation. This way there is no issue with preserving existing cues from the

situational data. The specific problem is transformation of situational indication

from one particular form to another familiar form, given an existing and detailed

domain description in the destination form.

This thesis uses the terms ‘data’ and ‘description’, to indicate forms of input

and output references respectively.The notions of transformation and indication,

are close in sense to the linguistic notions of translation and reference.

An aim is domain independence, with an open view that allows for multiple

interpretations. The basic notions, of emergence and reference-referent indication,

are drawn from philosophy and linguistics. In more conventional manifestations of

emergence, there is a transition from a system of lower complexity, to a system of

higher complexity.


This focus is on early emergence, with reference transformation intended to

improve the base from which emergence arises. There are constraints on the form,

of the emergent description reference. The complexity level, of the eventual result

of completed emergence, might possibly be of a higher complexity. However, the

changes that occur during reference transformation are more lateral, in terms of

complexity. During reference transformation, the pertinent complexity is that of

the situational referents. The transformation should be both domain and feature

extractor independent.

A system implementing reference transformation, will have access to a store of

experience, and attempts the role of the expert, or at least the expert’s assistant. The

thesis assumes unchanging knowledge and experience sources during interpretation.

A recipient of the transformed indication shouldn’t have to know about the

techniques used. Any data corpus used, in training or calibration, should have

been neither constructed nor expertly described with a transformation framework

in mind. Prior calibration should have be possible without involvement of the

expert whose descriptions are associated with the calibration data. An expert,

whose descriptions have been used in calibration, would have had no access to or

knowledge of the techniques that enable lateral reference transformation. The expert

would not have accessed feature data, such as that available to the interpretation

system. Calibration is not necessarily training. The implementation specifics of how

the existing knowledge was acquired is outside the scope of the thesis.

Self-organization, though an aspect of many emergent phenomena, is not helpful

when a particular form is required, as what emerges from a different representative

form cannot be guaranteed to obey a particular destination form. The framework

develops a technique for dealing with the aspect of undirected emergence.


Some artificial life approaches seek to influence or control emergence by indirect

interaction, and by understanding and exploiting emergence [QQL06]. A similar phi-

losophy is employed in this thesis (see Section 3.1.7) for the reference transformation

in early emergence.

The next chapter is the heart of the thesis. It presents the a theory of transfor-

mation, and a framework that incorporates the theory. It introduces a formalism for

discussing reference transformation from one known form of indication (the data) to

another known form of indication (the description). The subsequent chapters, will

consider approaches for implementing the framework, and explore the consequent

implications in the context of the domains of tomography and wildlife observation.

Chapter 3

Reference Transformation

The previous chapter introduced references and referents, and other approaches to

the problem of interpreting complex data. In this chapter, a theory for reference

transformation is discussed, and a framework consistent with that theory is devel-

oped. This chapter is divided into 3 sections.

The first section explains concepts needed in a framework for reference transfor-

mation. Referents and references are revisited, followed by a progression through

emergence, knowledge and other concepts underlying the model.

The second section more formally presents those concepts as components that are

used in the framework. The subsections of the second section can be conveniently

referenced from later chapters as well as from the third section.

The third section lays out the components into the framework proper. The

framework facilitates emergence for a system that uses it. Data and knowledge

entities take part in the emergence of meaning. The framework will allow for data

to be processed either all together or in parts. The subsequent chapter suggests a

possible implementation approach and situations.

48

CHAPTER 3. REFERENCE TRANSFORMATION 49

3.1 Concepts Underlying the Theory

3.1.1 Assumptions

Feature Spaces

There is no distinction in this theory between raw data and feature data. There is

data, which is part of the indication of a situation. A complete indication of the

situation involves domain knowledge and other context. The data is considered to

occupy a feature space. The data wholly or partly interacts during emergence. It

is a practicality, of the likely discrete nature of the data, that the data will also

interact discretely. Though, in later sections sparse data is discussed, there is no

necessity for the data to be discrete.

Domain knowledge and other context can also be considered as occupying feature

spaces, possibly overlapping the data feature space. However, for simplicity, it is

assumed that knowledge of each space (data and knowledge) is independently known.

Empty data or empty referent

It is assumed that the calling system can detect empty data as represented in the

feature space, and not request the framework implementation to interpret it.9

The boundary case of no referent, when the data does not indicate any mean-

ingful situation, is outside the scope of this thesis. There is a danger similar to

assuming a soccer ball on a field (See section 2.1.2), though the assumption is of

general non-existence rather than specific existence.

9The calling system can choose to posit no situation, or a situation to exist that agrees withgeneral domain knowledge in every particular, or a situation that is not-significant.


Existing Sources of Knowledge

Humans, who have access to more sophisticated biological systems, take many years

to build associations. To artificially build a comparable source of experience would

require not only a knowledge source, but also a systematic way of updating it

using newly acquired experience and verifying its consistency. An existing source

of knowledge is assumed. The issue of modifying an authoritative general domain

knowledge source is not tackled in this thesis. However, modification of a temporary

description of a situation is a central notion.

Non-Applicability of the Closed World Assumption

In order for the Closed World Assumption to be true, all knowledge must be known

a priori, so that anything not explicitly stated in the knowledge repository must not

hold. And that anything stated must definitely hold. This would mean that new

knowledge cannot come into being, and also that multiple, possibly contradictory

interpretations are not possible. The theory allows for situations that were not

anticipated at the time of architecting the system.

Relevance of Neighboring Data

Continuous or slightly varying data is assumed. This is with respect to significance.

There is an assumption that neighboring data regions contain related data items,

even if adjacent data items are not directly related in meaning or significance. The

work by [SJ99] gets the recipient to organize data into related groups. The groups

created allow inference of significance; neighboring data units within the same group

are relevant with respect to each other. There isn’t as easy an assumption possible

in the data neighborhoods envisioned by this thesis. However, a traversal of a data


region should not see as sudden shifts in domain (or as many shifts of situational

significance) as fabricated content (such as in film).

One Expert

There is assumed to exist an expert for whom existing domain knowledge and

system calibration is acceptably reconcilable. Any existing calibration of a system,

is considered to reflect the observations of a single expert, as if that expert had

bootstrapped knowledge into the system. This is regardless of the possibility that

knowledge acquisition might have been an accretive effort from possibly multiple

independent authorities.

Consistency of Interaction

The interaction of data and description, however it is achieved, is assumed to produce

the same results if prior data and knowledge sets are the same. Practically this would

involve consistent feature extraction (if any).

3.1.2 Different Forms Referencing the Referent

Both data and descriptions are ways of indicating things in the real world, or notions,

or combinations thereof. Data is the original indicator, and might not be easily

appreciated by many people. A description is a different indicator, which is hopefully

more accessible to more people. This thesis investigates a transformation of the form

of indication; a transition from data to description. The transformation occurs in

parallel, with many small descriptions composing a greater situational description.

Data domains are restricted to those with unscripted sequences of data. Au-

thored data is avoided; there is no attempt to generate representation of the author’s


intent. Authored data is susceptible to “the Kuleshov Effect” - a notion from film

theory - where the sequence ordering affects the meaning [DDN03][Kat91]

The transformation seeks to generate alternative references, which indicate the

same referent (situation or notion) as the initial reference set (data). The informa-

tion content does not change, in the sense that the same referent is to be maintained

from data to description. There can be information in the data that does not appear

in the system experience, in that information in the data indicates referents not

indicated by the system.

However, it is likely that the representation adopted as the description will

suggest a different situation to that indicated by the data. There will be both loss

and mutation of information during the reformation of the reference. An estimate of

confidence, in the reference transformation, is also an estimation of the discrepancy.

In many systems, knowledge is considered to be stored in text form, because

that is the way people are used to thinking. The form of their descriptions is that of

natural language; perhaps structured representations whose elements contain natu-

ral language. People associate referents with the words that indicate the referents.

Two representations will be considered equivalent if they both map to the same

natural language representation. Similar to the binding of attributes from different

database schemata to the one public attribute set.

Multimedia helps people view phenomena that they do not have convenient

access to. In some cases, the view is somewhat diminished compared to an in-

person observation. However, the ease of viewing compensates. It is also a way of

visualizing or thinking about an idea. It can present the data in a way that helps

understanding.

The referential space is capable of representing most situations within a chosen

domain, including situations not previously encountered. The destination form


needs to be sufficiently expressive to indicate a previously unfamiliar referent. The

referential forms vary with the domain; different domains will each have appropriate

forms capable of representing situations in them. Knowledge needs to be acquired,

in the right form, from the experience. Familiar phenomena can be normal or

abnormal. Familiarity relates experience, of previous situations, to observation of

current situations. Normality is a property of the domain. Similarly, unfamiliar

phenomena can be normal or abnormal.

Part of the motivation for this work is that, one day, domain knowledge will be

able be held in multiple media and forms, simultaneously. This would enable systems

to more flexibly analyze or inform, based on the context of a recipient, without

having to translate everything to or from a particular representation. However, in

this research the scope is restricted to transformation between two forms. Each of

the source and destination forms can be complex. The distinction here is that both

forms will have been known during the prior system transformation.

3.1.3 Keeping the Same Referent

The source and destination are different models of the referent. They both highlight

and obscure different aspects. The reference transformation seeks to convert both

the highlighted and obscured to differently highlighted and obscured in the other.

The benefit is in the new highlighting of what was once obscure.

Accessibility, by different recipients, depends on the form of a given reference.

Ideally, as references change, they keep referencing the same referent. A range of

referential forms can refer to the same referent. Even disjoint references can refer

to the same referent; even those of different media.


Though some information will be changed or lost during referent change, the

transformation of other information to be better highlighted can disguise the in-

formation loss, perhaps even improve the information indication, to the point that

there is effective information gain on the part of the designated recipient.

A bad choice of target indication form (not the specific configuration within a

form, but the form itself) exacerbates the problem, in that the likelihood of the

recipient missing some information increases, as well as experiencing information

loss. A good choice of target indication form will increase the likelihood that a

recipient will become aware of more of the situation. The choice of better form is

assumed to have been realized by prior experience.

The framework to be later introduced lays out the means of transformation from

a known worse form to a known better form. Even though a better form can be

known, the means of constructing an indication, of the same referent as indicated

in the worse form, is not necessarily known.

Prior experience of how general domain indication modifies in the presence of

alternate form specific information, is brought to bear through a framework. Prior

observation of re-formation of references from a less useful source form (deemed

harder to access by non-expert recipients) helps interpret current re-formation of

current references.

The means of indicating the meaning changes. The situation being indicated also

changes, but is more effectively described. In a useful transformation, the degree to

which indication of the referent drifts can still be useful even if it is non-trivial.

The referent itself drifts in the sense that there is a notion of a referent space in

which a location or a region of that space being indicated is not maintained. There

is no re-formation within the referent space.


Maintenance of referent is not something that can be measured directly. There

is no definitive attribute binding by previous agreement.

The associations established from prior calibration perform similar functionality

to binding. However, any authoritative descriptions of situations indicated by known

data are only really authoritative at the situation granularity; not at the granularity

of situational aspects or components. At finer granularity of either reference or

referent, there is no authoritative association. A lower level of granularity increases

the risk of misinterpreting implicit data references. It also increases the chance of

separation of references that need to co-occur in non-linear emergence.

The framework implementation receives data features, and attempts to map the

subsequent interaction results to the knowledge space. The knowledge space, K-

space, is the (possibly infinite) set of all possible constituents, configurations or

conformances that the destination reference form can take. Each state of emerged

reference has, associated with it, a confidence value. This is the confidence that

the emerged reference state indicates the same referent as the prior reference state.

Sufficiently high confidence, advocates presentation of emergent meaning. Further

drift, due to mapping back to K-space can either be simply accepted, or allowed for

in the confidence estimates. Meta-data can be kept with regard to the data features

that have thus far been interacted with each reference state.

3.1.4 Emergent Re-formation

A well conceived model will exhibit the complexity, and emergent phe-

nomena, of the system being modeled but with much of the detail sheared

away. John Holland[Hol00]


Traditionally, emergence is the phenomenon of the realization of a more complex

entity, from the interaction of less complex entities. This is the complexity found at

“the border of order and chaos”. [Wal92].

Analogously in the thesis:

• order comprises meaningful references to referents.

• chaos is comprised of unfamiliar data references.

• “the border of order and chaos” is the interaction between the data references

and the knowledge references.

Complexity is a notion of how sophisticated a situation or system is. Note

that what is meaningful, and what is unfamiliar, depends on the context of which

reference form is comfortable for the observer. A more complex entity is not

necessary; movement occurs laterally between forms that are differently complex.

The result is something that seems more sophisticated, clear or useful.

John Holland was the pioneer of the field of computational emergence [Hol00].

He had a class of models called Constrained Generating Procedures. Instead of rules,

he had mechanisms. A mechanism was a transition function. The thesis provides a

framework for emergence that performs a service analogous to Holland’s transition

functions.

The mechanisms or algorithms are domain dependent. Each domain or applica-

tion has to provide some components to the framework. An aim of emergence is to

free the architect of a system from having to anticipate all events in advance. The

benefit to interpretation is that unfamiliar situations are described, using the same

framework as the familiar. The unfamiliar aspects of a new situations are described

by emergent forms, though the confidence estimate will be based on the familiar


aspects. The situational data interacts with general knowledge to obtain specific

knowledge about the situation that the data is a specific reference to.

References are re-engineered, from one form, into another form.10. The source

reference is hopefully the best indicator of the intended referent, in that earlier

form. A system employing the framework does not try to ‘correct’ content of the

first reference. It assumes correctness of the initial reference. The referent indicated

by that earlier reference, whether intended or different to intention, will be the

referent maintained. Referent drift will be with respect to the initial referent.

In some domains, emergence as a transformation, results in conveyance of mean-

ing more convenient than manual labeling and are likely more meaningful than

arbitrarily assigned labels (which are often too brief). Unless of course any created

detailed descriptions are acceptable as long labels. Eventually, a mapping of suffi-

ciently rich description is needed; a construct by which the referent can be ‘pictured’.

Whether it’s called a long label or a description, is immaterial. The framework is

not concerned with the purpose to which the output description is put. Descriptions

are made without ‘awareness’ of the eventual use of the interpretation.

There are some drawbacks in emergent systems. One is that they require the

system to relinquish some level of control. Under existing types of emergence what

gets produced is possibly not what is required. This phenomenon is thought of

as emergent direction, and relates to referent drift which is discussed later in this

dissertation (see Figure 3.1). Creating a more complex form does not also guarantee

a form that is easily accessed or manipulated. The phenomenon is emergent form;

how the result is structured, and presented to the recipient. The confidence estimates

reflect both emergent direction and emergent form. Essentially, this is an attempt

10In the bigger picture, of improvement of references and their interpretation, the data will notneed to be replaced by description. For some domains, references that include the original data(possibly modified) will provide greater clarity.


to reduce the magnitude of the drawbacks, while not also reducing the benefits. The

confidence estimates are an attempt to say when the descriptions are trustable.

Kuipers’ terrain ontology [Kui00] provided a look up system accessible by a

remote robot, which tries to model its environment as it roves. The robot senses its

environment, attempts to make identifications, and consults the central repository

for suggestions of other objects that might be encountered. The thesis adopts this

approach of hypothesis and verification, during emergence and the organization of

multiple acts of emergence, to assist in modifying estimates of confidence in the

interpretation. The system is not required to obtain everything useful in one atomic

emergent leap.

It is possible that the data feature space has insufficient potential for partici-

pation of meaningful interaction. This is not something within the control of the

system. There is an assumption of sufficient knowledge in system experience and

sufficient representational sophistication in the feature space. The aim of the system

is to provide better expression, if there is hidden information to reveal.

3.1.5 Knowledge

Knowledge, as described in dictionaries, talks of significance and cognizance. To be

familiar with something; to recognize, to notice. Knowledge is used to recognize

familiar situations and to reason about new situations. When something is rec-

ognized, an association occurs between what is sensed or otherwise detected, with

a notion or some real world occurrence. Traditionally, a person is considered to

possess knowledge, if what they express agrees with what an authority ascertains,

or is internally consistent and plausible. In a more general sense, knowledge is a

model of the world that is consistent with what is observed.


Knowledge as Association

Communication of this model is part of the interface between the keeper of the

knowledge and the recipient. Between people, words constitute much of the com-

munication. Words are associated with past experienced events. Often an authority,

such as a dictionary or a teacher is employed to make sure that different people use

the same words for particular experiences. Experiences include actions, observations,

items and ideas. Authority is either experience with many people or a particular

person, and the experience is prior observation of that person, especially when

conscious signs or signals are made.

The descriptions used by an authority are references by that authority to prior

experiences. The description of situations are references in the context of those

situations. Descriptions, accessed from a repository or generated by a system,

that resemble authoritative descriptions will be taken as references to the prior

experiences associated by that authority or by others who have made the same

associations between those descriptions and similar experiences. If an authority

or the works of an authority are consulted in the formation of a repository, the

repositories constituent entities can be considered knowledge both individually and

when composed into a greater description. A calibration is assumed which correlates

the descriptions emerging from system observances of the same situations observed

by the authority, which prompted similar descriptions.

The thesis considers other forms of description than that of natural language.

The entities need simply be references to experience. Knowledge is an association

between two ways of referencing the same referent. For instance both the data and

the description are references to the same referent. The terms ‘data’ and ‘description’

are convenient labels for the source and destination references, approximating the

input to and output from interpretation.


Context is anything that influences the emergence, without being nominated

as either data or description. The domain of the data, is part of the assumed

context. It is characterized by a (possibly arbitrary) set of axiom-like assumptions

that happens to be valid for all situations indicated by observed data. The parts

of the knowledge source, that do not become part of the description, are part of

the context. There are knowledge elements that interact with the data elements (or

affect the interaction), and yet are not emphasized in the final description. This is in

addition to knowledge and data that would have influenced prior calibration. They

might not be particularly applicable in so far as the situation, but they represent

part of what the system knows about the world, and thus affects (or is affected by)

the emergence.

The constants and symbols of the Herbrand Theorem [BL04] have their analogs

in knowledge components and context. The contextual components are components

that are never individually realized. The context are the parts of the data which do

not get described. Domains are chosen whose situations can be more objective than

subjective, in an attempt to minimize non-data context. Entailment is mirrored in

emergence.

Knowledge is an association between different ways of referencing the same

referent. Both data and the description of that data are really references to the same

referent. The terms ‘data’ and ‘description’ are convenient labels for the source and

destination references, approximating the input to and output from interpretation.

Confidence in the output descriptions is based on prior calibration of emerged forms.

An initial system would have had to bootstrap initial knowledge or be given

access to the data of many instances of described situations. It is assumed that the

knowledge source is populated, with sufficient knowledge, for the needs of emergence.


Knowledge is notionally held in a way to accommodate the different forms of

representation used by the system. It needs to be held in at least a way that

complements the generation of the destination form. The knowledge doesn’t have

to be in the same form as the output. It can be associated with forms. Within a

representation of a situation, there can be many forms or instances of a form.

The form of the knowledge source is not necessarily measurable. A form that is

measurable occupies a measure space; two form instances can be compared based

on the dimensions of the space. A form that has no known measure occupies a non-

measure space. Rather than map a non-measure space to a measure space [SGJ01]

[YXL+03], a measure space can map to a space that can be a non-measure space.11

Emergent phenomena can be correlated with references in the description form.

Calibration is based on system experience, which includes prior observation of inter-

action. Calibrated transformation seeks to perform for other media what dictionaries

and grammars perform for natural language. Though confidence in calibrated

transformation is limited to bindings that might only be true for the authorities

whose descriptions were used in calibration.

System Experience

Knowledge can be a record, after experience (perhaps prior emergence), of these

associations. The association, however, can be complex (very non-trivial). In order

to get to the associated references, from the data, the system would have to capture

either the complexity, or the system state that gave rise to the complexity, or both

associated.

Any correlations of data, authoritative descriptions and emerged results, from

calibration, will have been stored as associations. These constitute the ground truths

and contribute to calibration. The basic methods of transformation are the axioms.

11depending on the domain and the chosen media and forms of the destination representation.


Comparison, of what is newly encountered and what was previously encountered,

provides the basis of the reasoning.

A given transformation has no particular significance without calibration. The

representation needs a sufficiently fine granularity, such that the mapping will find

a suitably fine set of views within the knowledge base, such that the reader of the

knowledge source can do something useful with it. This requires the knowledge

source to be sufficiently sophisticated and available.

Familiarity of the data domain is calibrated into the associations. The (human)

expert associates data and description. The confidence in future interpretation

is affected by the variety and granularity of the calibration data. The resultant

emergence of data instances, whose expert descriptions are the same, need to be

associated with the same knowledge source region or representation. Confidence is

explored further in Section 3.1.6 and Section 4.1.2.

Existing Data

The kind of calibration suggested lends itself to existing corpora of described data,

if those descriptions are in the desired destination form. There does not have to be a

special effort by an expert to assist in calibration. Though, metadata specifying the

source of the descriptions would probably be useful, as there is an attempt to limit

non-data context. The more differently sourced the calibration descriptions, the

less likely the plausibility of the descriptions of new situational data. Acceptability

doesn’t require an absolute accuracy to an absolute context. However, consistency

with regard to an arbitrary context is desirable.


Normality and Familiarity

The distinction between the normal and the abnormal, is different to the distinction

between the familiar and the unfamiliar.

For a technique to work the knowledge source needs to contain sufficient knowl-

edge of what is normal. The experts, who provided the descriptions eventually used

in calibrating the system, are more likely to remark on what is abnormal (as that

is what they are trained to discover). However, in order to diagnose abnormality

they have to know what is normal. While the technique is not supposed to replace

the expert, the technique needs be informed enough not to cause too many false

negatives (abnormalities are true negatives). Different sets of “normal” data can

provide tolerances, which will be used during estimation of confidence. A system

with greater knowledge, will have greater “richness of representation”. In turn it

will be able to specify more precise tolerances. Note that different sets of abnormal

data can also provide tolerances. Both normal and abnormal data can be familiar.

The same techniques for description emergence are used for both familiar and

unfamiliar. Unfamiliar phenomena are described with similar ability and confidence

as describing the familiar. Descriptions of what is unknown are either composed

from what is known and should be indicative of what is uncertain. Though the

uncertainty might have to be expressed for the overall situational description as

opposed to description of sub-regions of data. The confidence will be partially

based on how much is unfamiliar. It is possible that the system causes emergence

of a description of erroneously high confidence can emerge, if the estimation of

unfamiliarity is too inaccurate.


3.1.6 Confidence

Though interpretation of empty referents is not covered, an emerged new reference

can be empty. This would indicate that the system is unable to interpret the data. If

the emerged interpretation is accompanied by a very low confidence value, the system

would try to improve that value. If improvement does not occur, the description

will be no more useful than an empty reference.

Absolute vs Relative Confidence

Absolute confidence, in the accuracy of an implementation, is domain dependent.

So a thresholding approach to acceptable confidence is also domain dependent. A

transformation framework can help with increasing or decreasing confidence. Having

stated a threshold and calculated a confidence value, the system can decide to stop

if the latter is greater than the former. This is more an expression of comfort with

the interpretation.

Relative confidence, in the context of this study, concerns acceptability more

than probability. The system can choose to believe one interpretation is a better

indicator of the earlier referent, based on the confidence, even though it is not

necessarily a reflection of what is better.

Associating Confidence

Confidence is associated with an activity of emergence. While this could involve

traditional training techniques, the calibration occurs with regard the interaction

of data with knowledge, rather than just the data itself. The way the data and

knowledge interact is not yet calibrated within a completely fresh system. Meaning

is referenced by the resultant form. Confidence of meaning is estimated during

emergence. It is associated with past experience.


The richness of previously encountered data affects the confidence of the emerged

information. If exactly the same form is created from the interaction of different data

with the same knowledge and context, the different data instances are deemed to

have the same meaning or depict the same situation: indication of the same referent.

If the resultant form is sufficiently similar, it can be considered ‘close enough’. Here

confidence is inversely related to deviation from the data referent. The deviation

from normal or familiar interpretation provides information.

Improving Confidence

Initial estimates of confidence do not necessarily limit final confidence levels. The

seed data (that which is initially interacted) might not possess salient data indica-

tors. Further traversal of the remaining data, during iteration seeks to encounter

more significant data or otherwise realize the emergence of more significance from in-

teraction. Confidence in interpretation hypotheses can be strengthened or weakened

as iteration progresses. Each subsequent interpretation is a hypothetical reference

of the data referent, and the confidence reflects acceptable referent drift. Change in

confidence of a particular hypothesis could result from combination of confidence (for

example, by Bayesian statistical methods) from multiple hypotheses, interpretation

by other means of data that has already been interpreted, or interpretation of further

data.

3.1.7 An Indirect Approach

Direct interaction of entities sees a resultant structure being created from an oper-

ation or function with the participant entries as the only operands. Direct transfor-

mation requires modification of a source form or representation into a destination

form.


There exists a problem, with regard to the results of data space entities directly

interacting with each other. There is very little chance that a useful and familiarly

accessible knowledge representation will be formed. A solution is to accept whatever

representations emerge as useful, and to calibrate the system with respect to them.

However, this still does not guarantee a meaningful knowledge representation that

indicates a situational referent.

Another solution is for the feature space entities to indirectly interact via another

entity. Though this seems to just replace the original problem with interaction

between feature entities and the ‘other’ entity. However, it can be of a represen-

tational that is similar to the desired form. It will not be exactly the same as

the destination form, as it needs to be able to be manipulated or perturbed. It

forms a bridge between indirectly interacting data entities. It also provides a bridge

between the initial domain approximation of a situation, and the final more specific

indication of the situation. Also, it means that there need only be O(m) mechanisms

of interaction if there exist m types of data reference. This is much reduced from

the O(m2) mechanisms required if the data references interact directly.

If the feature entities interact with the knowledge representation and simply

perturb it, the perturbed knowledge representation can be read. This still has

the issue of whether the perturbed form remains representative of the referent and

accessible by the recipient. If two (or more) feature entities interact and perturb

the knowledge representation, both (or all) are considered to have perturbed the

knowledge representation. There is an issue with non-linearity. Parallel perturbation

notionally applies both interactions simultaneously. The effect due to one interacting

feature entity has to be influenced by the other interacting entities, if there are non-

trivial non-linearities. Indirect interaction of the feature entities desires simultaneity

of interactions.


The form that is perturbed is seen as an intermediate form. The entity (of this

intermediate form) will henceforth be known as a “bridging entity” (denoted B).

The information regarding the situation referenced by the data, is present in the

perturbation of the bridging entity. It is this perturbation that is calibrated; the

bridging entity is also considered a knowledge form. There will need to be mappings

from the space of possible destination knowledge forms (K-space) to the space of

possible bridging entities (B-space). The only augmentation of the inhabitants of

B-space should be the ability to be manipulated or perturbed.

The intermediate form is associated with the destination form. There can be

many intermediate forms or many instances of the same type of form, as long as

there is an association with a final instance of the knowledge representation.

The approach requires a useful knowledge representation of the domain knowl-

edge, and have that perturbed by interaction with the data. The interaction is seen

as indirect interaction between elements of the data, as well as interaction of the

specific situational with the general knowledge.

There are trade-offs to having an expansive domain description subject to local-

ized perturbations. Different data portions can be separately interacted. Indirect

interaction of features allows for capture of non-linear effects. However this only

manifests amongst data presented for concurrent interaction. Partitioning of the

data helps rein in exponential growth of computation with interaction of data

features (indirectly) with all other data features and (directly) with the entire

domain description.

3.1.8 Composition

Referents can have multiple references indicating them, and they can be composed

of sub-referents that can in turn be referenced by many references or combinations


of references. The transformation systems manage knowledge, context, knowledge

of prior emergence, references and the method of transformation. If a particular

future form of reference is desired, it needs to be found in the composition of the

other transformation system entities and/or naturally emerge.

Both references and referents can be composed of multiple entities, including

other references and referents respectively. With referents being intangible, in the

context of transformation of references, an overall referent is considered situational

and sub-referents are associated notions.

Composition of referents is presented in theoretical terms to explain how emer-

gence in transformation works. The composition of references however is something

that can be a resultant output of transformation.

There is no referent delivered to the recipient. The references that the recip-

ient sees conjure association or comparison with their personal experience. The

juxtaposition of multiple references might cause them to conclude a meaning that

is a combination of their own referents. To deliver appropriate references to the

recipient, the transformation system will model referents via transformation system

experience, say in the form of associations in the knowledge source.

The emerged references can be multi-valued, as the activity of emergence can

throw up different possible interpretations. It is possible that later some candidate

descriptions are removed due to low confidence, leaving only a few (possibly only

one). To some extent this pruning will be dependent on the domain knowledge.

The further composition of references, is the transformation system analog for

reasoning, and can occur in several ways. Simple bulk presentation of multiple

generated references is the easiest.

Combination of results is not the focus of this thesis; existing AI combination

techniques can be used, if no further emergence is required.


Partial Descriptions of Data

There is no advance identification, of which parts of the data are salient. This

is improved on, if either emergence occurs from multiple finer references (if, for

example, the data is naturally and conceptually partitioned) or multiple concepts

can be distinguished within the emergent form. These partial description fragments

are used similarly to words in a natural language. The collection of multiple desti-

nation form fragments might be determined by the recipient to constitute multiple

concurrent situations or as composing of a single description. This is different

from sewing together several little classifications of small data packets processed

separately. The data is still processed together. If the transformation system

individually and confidently detects many concepts, a situational data referent can

be referenced in greater detail. Associations are made between the emerged results

and the knowledge source.

If the knowledge source is granulated finely, the emerged result can be described

in more detail when associations are made between emerged entities and the knowl-

edge source. The information from prior calibration is stored in an auxiliary bridging

entity denoted by B∆. Calibrated associations are stored in B∆; they assist in

mapping back from the perturbed bridging entity in B-space to K-space references.

However, uncertainty is greater for finer granulation, as existing calibration will have

been most valid for the situational level. Tolerances, will have been qualified as

based on assumptions of similarity in calibration data instance significance. At finer

granularity, only certain details of calibration data might have been described. The

same errors or variances would exist, but would be proportionately larger. There

is confidence of combination along with the combination of confidently expressed

concepts. More uncertainty is introduced with confidence in combination, and

combination of confident descriptions.


Though it is possible to discover or reveal complexity from seemingly simple

systems, generating sufficient complexity, allows the use of a resultant form as a

means of communication. There may be multiple destination referents indicated.

Typically, there is no need to know how many individual referents there are.

Uncaptured context

A situation is considered to be indicated by the data nominated. Knowledge is

available in the experience stored or reconstructed. There is a problem that not all

interactions will occur, even if all significant entities are available. Context involves

the notion of influences, on the interpretation, which weren’t nominated as either

data or knowledge. This includes how the entities interact. It is a way of thinking

about loss in the emergence processes.

Cross-topic Interference

Generally, to realize better emergence, interaction is desired between entities not

normally seen as being related. Information can be hidden in the interaction of

co-occurring data.

A situation may contain different referents (even given the same data and con-

text). The activity of emergence of a reference indicating one of those referents,

might interfere with the emergence of a reference indicating another referent. This

is analogous to destructive interference of electric signals. One topic becomes a false

context for another topic.

A containment of cross topic interference is attempted by reducing the amount

of data or knowledge available for each activity of emergence. The idea is to

create usable resultant references, and to recombine them later, with multiple topics

described. This is dangerous as it depends on the combination of emergence results


to salvage sophistication or nuance. Information, otherwise due to non-linear inter-

action, can be lost. Where to partition data and knowledge is a difficult problem, as

our techniques are trying to capture the situational information that is not obvious.

If data is decomposed into smaller units there is a danger of losing existing non-

linearity, if the non-linearly composing entities are separated. The advantage of

having naturally decomposed data, is that it is easier to deal with noise, and there

is a lesser potential for cross-topic interference.

This will often be manifested in partitioned data neighborhoods, allowing for

data to be fed to interaction portion by portion. The pre-decomposed nature of the

data means that any loss of non-linearity has occurred before the data was created

or captured.

Though reference transformation attempts maintenance of the data referent, the

focus is on differently formed references that are better in terms of comprehension

more so than content.

Whether by cross-topic interference or decomposition loss, there is expectation

of inaccuracy or loss in description. Part of an emergence transformation system is

aimed at addressing this loss. Where possible, decomposition is decreased to stem

loss of non-linear indicators. Decomposition is increased to decrease the effect of

both semantic and random noise.

3.2 Reference Transformation

This section lays a more formal foundation for the concepts discussed earlier in

the chapter. The general knowledge is a description of what is known about the

domain, a representation in the same desirable form of description accessible by the

recipient. The representation of general knowledge can also be notionally considered

an approximate description of the specific situation. As the framework is employed,


general domain indication is progressively transformed into specific situation indi-

cation. This reference transformation sees the modification of part of the general

knowledge to include a description of the specific situation. The environment in

which this change occurs is predominantly the situation data. The environment

also includes prior calibration which reflects some context. The regions unchanged

will notionally describe the parts of the general knowledge which still hold true. The

confidence in the approximation can be poor if the specific situation differs greatly

from the prior experience of the general knowledge.

If the general knowledge is sufficiently complex, interaction with the environment

should give rise to a better approximation of the specific situation.

3.2.1 Different Forms Referencing the Referent

Referent Equality

The destination form is considered better capable of representing the same infor-

mation than the source form. It is called the description because it is considered

to provide more accessible information. More accessible information is considered

more explicit in so far as it is less hidden from the recipient. D represents all

the data available before emergence. All references, configured in the destination

form, are considered to inhabit a knowledge space or K-space. K0 is the initial

knowledge available (the domain knowledge) for interaction with the nominated

data. Context is anything that will influence the transformation, and which is

neither the nominated data nor the domain knowledge. The notion of knowledge is

that of accepted collective references in, or renderable into, the desired form.

KD is the ideal reference in K-space that would indicate the same referent

indicated by D. The actual reference formed after emergence is denoted Kfinal.

Kfinal also indicates a referent; hopefully the same referent, but not necessarily.


An overall referent is denoted R; the referent corresponding to D is RD. The act,

of indication of the referent by the reference, is represented as R (D). No referent

RD is actually presented to the recipient. This is because R (D) is the indication

experience of an expert looking at the original data. The R (reference) notation is

also used for intermediate references during emergence.

RDRKfinal

DKfinal

K0

indication

drift

Figure 3.1: Referent drift as knowledge form changes

The description is a modification of the initial knowledge. The discrepancy

between the ideal referent RD and the final referent RKfinalis conceptualized as a

drift from RD to RKfinal. This drift is estimated as a confidence XKfinal

.

XKfinalis the confidence associated with the notion

R (Kfinal) = R (D) (3.1)

There is no measure of the change in referent. There might also be no direct

measure of the change in reference, as D and Kfinal are of possibly different form.

For the data to be partially included in the description, the knowledge form would

need to bear some similarity to, be able to embed, or be otherwise compatible with

the form of the data. The generation of a confidence estimation value is used instead.


Forms and States of Reference

Confidence is also used in a hypothesis-test cycle or iteration. The aim of the

iteration is to be able to regularly check confidence in Kfinal as it emerges. It also

provides an ability to check data portions gradually.

The collective reference to RD is denoted ρ. ρ comprises the data, the current

knowledge, and any context.

ρ = [D,K] (3.2)

The collective reference at stage n is denoted ρn. The transformation from ρn−1

to ρn is denoted

ρn = Ψ(ρn−1) (3.3)

where Ψ represents the invocation of transformation that occurs during a single

iteration.

ρ0 contains the data (D) and initial knowledge (K0).

ρ0 = [D,K0] (3.4)

Similarly, for other reference states ρn there is

ρn = [Dn,Kn] (3.5)

where Dn is the collective data (d0,...,dn−1,dn) that have thus far been involved in

interaction with knowledge representations.

As in equation 3.1, the aim is for

R (ρn) = R (ρn−1) (3.6)


Each referential stage is associated with a referent; in the best case each new referent

will be the same as the one before.

Consider the transformation from an initial state to a final state (an interpreta-

tion):

ρ0, ρ1, . . . ρn−1, ρn, ρn+1, . . . ρinterp (3.7)

Each step, ρn to ρn+1, is an activity of emergence. This series, of invoking

transformation, occurs until a termination criterion is satisfied. one criterion is an

observation that subsequent reference stages will not get appreciably better; that

an equilibrium has been reached. Alternatively, the invocations can be terminated

when some threshold condition has been reached. An example threshold condition

is

X (R (ρn) = R (ρ0))≥Xthresh. (3.8)

Confidence is checked after each iteration.

ρinterp is the system interpretation (a representation comprising other compo-

nents; composed and presented by the system). This system interpretation is given

to the recipients; they can interpret further if they so desire.

Ψ is the transformation operator, which is overloaded on the types of the entities

taking part in synthesis. Different algorithms can be used in implementation for

different types of entities.

ρn = Ψ(ρn−1) (3.9)

Ψ signifies active or implied progression toward overall emergence. As applied

to multiple knowledge fragments it means composition. As applied to reference


state, ρ, it means invocation of interaction. In both situations emergence can be

continuing. Emergence is not necessarily invoked by interaction. It is something

that occurs of its own accord.

It’s possible to have multiple situational interpretations indicated by the data.

So rather than simply R

R1,R2, . . .Rm, if ∃ m situational referents.

R1,R2, . . ., if ∃∞ interpretations.

The multiple situations can be represented by multiple final reference states.

Each situational referent per situation referent could be composed in the mind

of the recipient using several smaller sub-referents. Each sub-referent could be

indicated by one or several sub-references from ρfinal. The sub-references can

overlap; the sub-referents can overlap. Even if only few situational referents exist,

it is unlikely (in most domains) that a transformation system would confidently

claim the event of having provided interpretations for all situational referents. The

transformation system does not predict, let alone indicate, the number (possibly

infinity) of situational referents. The aim is acceptable interpretation; not the best

interpretation. Acceptable interpretation can include a single interpretation of one

of many co-existing situational referents. ‘Acceptable’ is provided to the system in

the guise of Xthreshold.

If iteration terminates at ρn, then either ρn is an acceptable reference state,

and/or no further improvements are likely to be made. ρn could be an acceptable

reference state because of confidence thresholding where equation 3.8 holds. Alter-

natively, it might or might not refer to the same referent, as it would have reached

equilibrium, and no more improvements or changes are likely to occur.


3.2.2 Deviation from, and Convergence to the Referent

Though the aim is to maintain indication of the initial referent, a deviation from the

referent is expected. The technique used for transformation can range in accuracy

from ideal to very limited.

Rn = R (ρn) (3.10)

Rn+1 = R (ρn+1) (3.11)

A system that converges Rn+1 to Rn might chain several transformations. In

the trivial case of the data being both the same form as the knowledge source and a

subset of the knowledge source instance, the system should terminate, perhaps with a

record of which subset of the knowledge source the data is. Given a situation familiar

to the knowledge source, but with data of a different form, in principle, Rn+1 should

converge quickly to Rn. Situations which are not familiar to the knowledge source

but have familiar subcomponents, might take longer, as X (Rn+1 = Rn) might start

at a low confidence level; more transformation steps required. Situations containing

events unfamiliar at component granularity, will receive one of two broad system

responses. The system will either present the results of its reasoning, or the system

might state that it is a new situation that needs to be looked at more closely by an

expert.

Drift

The domain knowledge K0 for the purposes of this research is considered unchanging.

It is copied to make an initial version of a bridging entity B0, which is allowed to

be modified. Emergence occurs as D is progressively interacted with the bridging


entity. In order to be modifiable, and also because the result of interaction might

not stay within the knowledge space K-space, B-space can subsume the K-space.

Beyond natural decomposition of the available data, D, it is not necessary that the

features be distinguishable or discrete, so long as they can be interacted with the

bridging entity. Issues of sparse data are explored in Chapter 5; some significant

data can exist in data portions not yet reached, when termination of iteration occurs.

Figure 3.2 illustrates that the final bridging entity will need to be projected back

from B-space to K-space.

RD RBnRKfinal

D

K0 B0

Bn Kfinal

Figure 3.2: Intermediate step of modifying a Bridging Entity

Each intermediate entity, as well as D and Kfinal has a corresponding referent

which notionally occupies a position in the referent space. As with figure 3.1, the

red lines in these diagrams signify indication of a referent by a reference.

n−2

Rdi

i=0

RBn

dn−1Bn

Bn−1

Figure 3.3: Referent drift during a single iteration

With each new state of the bridging entity, Bn, the indicated referent will drift

away fromn−2

Rdi

i=0

. In figure 3.3n−2

Rdi

i=0

is the collective drift due to the previous n-1

interactions with data portions d0 through dn−2. It can be hoped that some of the


drift will be a drift back toward the data referent. In any case, it is the task of

confidence estimation to gauge overall drift from the data referent RD.

Confidence

•

•

XBn−1

XBnδX(Bn−1,Bn)

Xthresh

Figure 3.4: Improvement past threshold confidence from time t1 to time t2

In figure 3.4, Xthresh is a threshold confidence determined during prior calibration.

The interaction phase will terminate upon estimated confidence surpassing Xthresh.

The framework attempts produce an acceptable interpretation, not necessarily the

best one.

•

•

Xthresh

•

•Xthresh ••

Xthresh

(a) (b) (c)

Figure 3.5: Confidence below calibration threshold but changing.

It follows that most of the time, the confidence estimation of the current reference

state will be below threshold. If there is a consistent trend is toward Xthresh, it

would be just a matter of waiting for Xthresh to be surpassed. However, there exists a

second trigger for interaction termination. An estimate that the confidence level has

leveled off, is in equilibria or if in fact referent drift seems to overhauling the efforts

of further interaction. Though the absolute confidence estimation in figure 3.5(a)

might be lower than the confidence in figure 3.5(b), termination is more likely to


be triggered in situation (b) than (a). The resulting confidence, which is below

threshold wouldn’t be considered acceptable. Though, depending on the domain,

the description might be still presented with a warning. The framework would be

cutting its losses even if close to the threshold level. The trigger doesn’t simply have

to be based on comparison of the immediate pair of past estimates. Trends could

be determined, where even a dipping transition, such as figure 3.5 is considered a

temporary event.

3.2.3 System Experience

The bridging entity B∆ is the store of system experience. The system needs, in

effect, to cause interaction between the data and a knowledge source. As data (D)

and knowledge (K) are of different forms, the significance of the interactions are not

known axiomatically.

To help interpret the possible third form that is the result of the interaction,

prior system experience is required. This way the significance of measurements,

from emergence, can be extracted in terms of the knowledge source - our description

K0. This experience is modeled as a calibrated bridging entity (B∆). B∆ contains

mapping information, from B-space to K-space that would have been established

during calibration, when B0 would have interacted with calibration data. This

information is employed upon the completion of interaction of B0 with new data.

B∆.(mappings) = Γi(bi) (3.12)

KDcalib

iis an authoritative description of the situation corresponding to the ith

instance of calibration data involved in emergence. bi is an association, binding

KDito the interaction of Di and Ki. These are the ground truths of the system. Γ


is shorthand for the collective information from multiple prior binding associations,

bi.

bi = [KDi, Ψ(ρi)] (3.13)

Ψ(ρi) is the synthesis of the constituents of ρi; This should not be taken to be a

synthesis of ρi−1 and ρi, where the latter is the successor of the former.

B∆ = Γi([KDi

, Ψ(Di,Ki)]) (3.14)

The bridging entity stores the combined information from the knowledge source’s

associations of data and knowledge. Measurement from the bridging entities interac-

tion with new data, can result in the estimation of the description of DnewKnowledge.

The bridging entity, B provides a way of discovering aspects of, and augmenting,

the destination form, that can be manipulated by the framework. It is an abstraction

of the knowledge with special attention to association.

Tolerances and Confidence

To make sense of the results of interaction, there is a need for a calibrated system

that associates meaning with similar, if not those results. Multiple calibration

descriptions can be accepted as being bound to Ki. Many emergent events, possibly

varying in form, can be associated during calibration to partial forms of km of

Ki. Γ needs to handle variation in calibration information. This variation is the

basis for the tolerances the system uses to estimate confidence. The information

about confidence will depend on the domain; this information can be stored in the

bridging entity. The confidence estimate in an interpretation of new data, will be

seeded from B, and will be raised or lowered by verification with other data or in


combination with other estimates. The confidence is not stored in the knowledge

source as the different experts and recipients accessing the knowledge source can have

different contexts. Calibration of a bridging entity for each purpose is conceptually

easier, than attempting to describe context or handle context variation within a

knowledge source. There is an expected redundancy in variation information held

across different contexts, as many variations will be similar across different contexts.

The different contexts possible, owing mainly to different sources of descriptive

authority, will result in multiple B∆ entities. Though only one B∆ would be used

during an iterative emergence.

There will be many possible recognizable or acceptable subregions of B-space

which are interpretable B-space configurations. These fall into two main categories:

configurations already present in B∆ or configurations that can be inferred from

B∆. Other configurations might not be interpretable. Every iterated version, of the

bridging entity Bn, is an approximate expression of reference to the data referent

R. Bn might only be a portion of the results produced from the interaction of

data with Bn−1. B∆ is stores information from prior calibration that is used to

lend significance to part of the results of interaction. In the event that there exist

configurations in the results that do not exist in B∆, hypotheses could be created

pending later verification. B∆ can be seen as cleaning the results, of interaction, so

that what remains is an interpretable Bn+1. This enables Bn+1 to be used by the

next phase (see section 3.3.1).

Returning to the Chapter 1 analogy to identifying constellations. The constel-

lations are groups of stars, in the night sky, with names mostly from mythology.

The largest and longest constellation is Hydra, the multi-headed sea-beast, which

used to include the constellations Corvus, Crater and Sextans. Other societies

have made different groupings of the stars, and called them according to their own


nomenclatures. The original Hydra contained the stars left over after the other

constellations had been named. The previously named constellations are analogous

to the familiar configurations in B-space. The original hydra is analogous to the

parts of the perturbed bridging entity that do not correspond to calibrated parts of

B∆. The breaking into smaller constellations, would be analogous to interpreting

the unfamiliar parts as multiple configurations.

3.2.4 Decomposition and Composition

There will be individual reference elements, k, that compose Kfinal. Individual

referent elements, of a collective referent, will correspond to different elements of

the situation. During prior system calibration, an expert description of a situation

would have been of the collective situation. Individual ks do not necessarily map

1 : 1 to individual referents in an overall referent.

Decomposition and Data Neighborhoods

If the data is already and necessarily decomposed, such as the cases of tomographic

sections or video frames, then the benefits of decomposition can be taken advantage

of without additional unnecessary reduction of the data. Data neighborhoods also

enable verification and augmentation of an interpretation, by activating emergence

in some sections of the data, and also in neighboring sections. Emergence can occur

from several points in a data neighborhood in parallel. This is analogous to a person

taking in the entirety of a scene in a glance.


Fα

Fβ

emergencekα

kβ

indication

r1 r2

composition

R

Figure 3.6: Composition of an overall referent, from partial referents

Composition

Composition covers the combination of multiple referents into one situational ref-

erent, the synthesis of unfamiliar situations from familiar components, and the

synthesis of unfamiliar situations from inferred components.

An overall situational referent could be comprised of multiple notions. In Fig-

ure 3.6, each of the two notions, r1 and r2, could be indicated by a separate

description, k1 and k2 respectively. The benefit from the framework is simply the

transformation of each separate group of features into the descriptive form.

Composition can also be involved in the formation of references. Figure 3.7(a)

considers a non-linear composition. Though it is a composition in the realm of the

recipient. The presence of the multiple references (e.g. bird cries and deer scatter)

is sufficient to indicate the concept in r (e.g. predatory presence). Figure 3.7(b)

illustrates the situation where non-linearity occurs during interaction. There is an

appeal to decomposing the data, with the hope of later re-composition. Though a

mapping between reference items might not be adequate if non-linear references are

decomposed by linear separation, prior to transformation. In Diagram 3.7(c) The

selection of data, for interaction, can have an impact on the situation suggested.

Multiple situations can co-exist.


Fα

Fβ

emergencekα

kβ

indicationby multiple references

r

R

(a)

Fα

Fβ

non-linearemergence

Kγ

r

R

(b)

FαVarying

dataselection Fα,Fβ

Kα

Kγ

Different indication

r1 r2

Different referents

R1 R2

(c)

Figure 3.7: Different types of composition


Cross-topic Interference

The possibility of multiple situations, even if handled intangibly at composition,

makes it possible for some or all situational references to poorly formed.

Fζ

Fξ

Fη

kζ

kξ

Kη

kζ

kε

r2r1 r3 ?

R2R1 R3 ??

Figure 3.8: Cross-topic Interference

Given that a situational referent can have multiple sub-referents (see Figure 3.6),

which might not all correspond to the one topic, it is possible to have multiple

candidate hypotheses within the one context.

On the left hand side of Figure 3.8, the data is present in the data feature groups

of (Fη) and (Fζ , Fξ). The data groups have sufficiently low overlap so that both

situational referents RR1 and RR2 can form properly.

r1 = R (Kη) (3.15)

r2 = R (Ψ(kζ,kξ)) (3.16)

Ψ in equation 3.16 signifies the implied composite reference (Kfinal) transiting

from the combination of kζ and kξ. Indeed it is Kfinal that is presented to the

recipient.


On the right hand side, the data is non-trivially intertwined, so much so that

the interaction of Fη interferes with the interaction of Fξ; resulting in an erroneous,

garbled or inaccurate description kǫ. Fζ modifies a localized part of bridging entity,

giving rise to an incomplete situational description.

3.3 Emergence Framework

This section puts together the framework which is the focus of this thesis. It enables

emergence by bringing together the concepts introduced earlier in this chapter.

There are two stages involved in the emergence (see Figure 3.9). The first is an

iteration of perturbation by interaction. It uses tangible records of transformation

system experience. The second stage begins upon termination of iteration. It calls

upon the possibly intangible experience of both the transformation system and the

recipient.

K0 is the existing domain knowledge, in K-space - the form in which the final

interpretation Kfinal will be presented. B0 is essentially a copy of the domain

descriptions of K0. B∆ holds calibration information, including both tolerances and

mappings.

The initial knowledge source, K0, pre-existed B∆, and is independent of it. How-

ever, the experience in B∆ is based partly on K0. For simplification, the subscripting

of bridging entities does not indicate K0 because all emergent interpretations are

expressed in terms of an unchanging K0.

3.3.1 Iteration

Interaction occurs until confidence in the emerged reference state is high. The test

of confidence can occur at the stages of either B-space entities or K-space entities.


OUT OF SCOPE

IN SCOPE

d1 dn−1d0

B0 B1 B2 Bn

Final Interpretation

. . .

Kfinal

. . .

B∆ mapping

B∆ confidence estimation

Interaction of dn with Bn

. . . Selection of dn from D

D Data Features

∗ Knowledge formations in K-space

⋆ Knowledge formations in B-space

Formations presented

Composition⋃

kformation )

Formation manipulation and re-organization

Figure 3.9: Emergence Framework

Modeling of those entities as the descriptive K-space references, in Kfinal, begins

after iteration termination.

The inputs to the framework are the reference data12 (D) and the knowledge

resources (which can be found in multiple forms: K0, B0 and B∆). B0 is an

augmentation of K0, enabling perturbation by interaction. B∆ is B0 with additional

information from calibration. It is this extra information that enables us to interpret

the results of interaction of the situation data D0, with the bridging entity B0. The

interaction is always between part of the data dj with the current version of the

bridging entity (Bn).

12to a particular situation


Reference State

The purpose of the framework is to transform the initial reference (the data), to

a differently expressed reference (the description). Although referent maintenance

is attempted, some deviation from the data referent is expected. In figure 3.10 the

transition of reference state is accompanied by a drift with regard to what referent is

indicated. Ψ signifies a transition of reference state, by invocation of transformation.

Rn Rn+1

ρn ρn+1

Ψ

Figure 3.10: Transition of Reference State

The framework is comprised of the entities which are references, and places for

the mechanisms that transform those references into other forms. The final group of

references is considered a group of knowledge elements which compose one or more

candidate hypotheses. The recipient of Kfinal may determine whether there are one

or more situations. No attempt is made by the framework to artificially partition

the descriptive references into references of different situations. When emergence

is triggered in parallel from multiple data locations, there will be exploration, via

interactions, in data neighborhoods around seed location. The neighborhoods can

overlap. There can be multiple hypotheses; each hypothesis stems from a seed. Some

hypotheses might be removed by means of confidence checking during iteration.

The activity of emergence is invoked multiple times with the aim of progressive

formation of descriptive references. It is characterized by the transformation

ρn+1 = Ψ(ρn) (3.17)


This equation denotes the transformation from one referential state to another.

Ψ is shorthand for the transition of the constituents of ρn into ρn+1 by invocation

of emergence.

Multiple invocation of interaction effects a chain of these transitions, seeing a

progression of modified reference states (See Section 3.2.1)

ρ0, ρ1, . . . ρn−1, ρn, ρn+1, . . . ρinterp (3.18)

Termination of this chain is discussed later in this subsection. The framework

encourages interaction until elements of sufficient quality have been selected. If the

situational data description does not vary from the domain description, it might be

difficult for the recipient to determine this, unless there is an understanding that

the differences between ρinterp and ρ0 will be emphasized.

Domain Knowledge

The domain knowledge from the knowledge source K0 is also fully present in the

initial bridging entity B0. This is the first approximate description reference to the

situational data referent.

The bridging entity, B∆, contains information for the calibrated interaction of

elements from both D and K forms. This calibration is expressed in the form of

tolerances, which in turn are the basis for confidence estimation.

Specific Situational Data

In the simplest case, each dj is the same as D. All the available data is used in the

next step - interaction. This is the best scenario if, later, the estimated confidence

level Xn+1 is sufficient. There are two alternatives for the simplest case. Possibly

there is only one invocation of emergence interaction; no more data is required.


Otherwise there is more data available after each invocation. This can occur in

either or both of two scenarios. The results of the first invocation provide directions

to the (outer) system of what data to get and where to get it. Alternatively, there

is a steady stream of data and the system uses whatever it has at the start of each

invocation.

For the first alternative, n = 0 only, so Dj = D0 = D, which is the data

initially possessed. Also dn = D. The situational referent being described is the

same as D0. For the second alternative, the situations being described are those of

D0,D1,D2, . . ., where each subsequent Dj subsumes Dj−1. ∀n,Dn subsumes D0.

This raises the issue of whether R (ρ0) is being preserved in each R (ρn). This can

be seen as an extreme case of a data neighborhood, where the framework has only

one seed position it can try.

In other than the simplest case, the system needs to decide which sub-corpus of

D will be used as dn. When D has no subscript, it is the usual consideration of the

situational data being all available before iteration.

Ideally, the choice of dn will be obvious from the structure or organization of the

data. Less ideal is a random selection of dn from Dn. This is also the case with

arbitrary selection. A semi-arbitrary selection is to exclude D0 from D1. This is

analogous to set difference ({A} - {B}). (D1 −D0) is used to denote the data from

D1 that is additional to D0.

Transformation of D1 − D0 (Dn −⋃n−1

0 Dx in the general case) can incur a

contextual error, especially if D0 is large compared with (D1 −D0).

When all data is available from the start, emergence can be performed on each

of differently seeded sub-corpora with each one being a different hypothesis. For

example, di and dj are different, possibly overlapping, sub-corpora of D. When di

is used as the seed d0, dj will be used in a later dn. When dj is used as the seed


d0, di will be used in a later dn. This way, there will be multiple paths through

the same data, with the advantage of being able to compare the systems different

hypotheses.

Interaction

Bn is the current state of the bridging entity B. Bn+1 will be the interpretation, in

B-space; the result of the interaction of Bn with dn.

(B − space) Υ D → (B− space) (3.19)

B0 is a special case in that it occupies K-space as well as B-space. B0 mimics

K0 in that it is a copy, augmentation or an abstraction of K0.

(K − space) Υ D → (B− space) (3.20)

In a typical interaction:

Bn Υ dn → Bn+1 (3.21)

dn is more complex than a scalar or single value of an attribute. It is differently

complex than Bn. dn is a feature sub-collection from the data source D.

Υ notation is used more extensively in Chapter 4.

Let dn = 〈fnx〉, where f denotes feature entities that are able to interact with

the elements of Bn. The size and composition of 〈fnx〉 vary with n. The interaction

dn Υ Bn can be alternatively denoted 〈fnx〉 Υ 〈bny〉. To avoid confusion with

formal set theory, ‘〈’ and ‘〉’ are used instead of set notation; subregions of spaces

( B-, K- and D- ) are dealt with instead of subsets of sets. Organization might

matter, depending on the knowledge source and data. Let Bn = 〈bny〉, where the


size and composition vary with n. Υ captures all interactive relations between

〈fnx〉 and 〈bny〉. If there are non-linear or structural relations, it is more than just

〈fnx〉 Υ 〈bny〉 = (f0b0 . . . f0by . . . fxbx . . . ). The recipients of the system can provide

different ways that B-space items and data items interact; they do not have to

manually provide meanings for the possible resultant values. The results of the

interaction can be of different type and more complex than anything encountered

in dn and Bn.

Several alternative hypotheses, might be spawned at this point, to be tested later.

The reduction into sub-configurations has issues with loss of non-linear emergence.

The way, that reduction occurs, is dependant on the technique used to represent

B-space. Chapter 4 investigates one possible representation of B-space.

Simultaneity of interaction is required for the indirect interaction of data during

emergence. In the sub-neighborhood of situational data dn being interacted with

bridging entity Bn, a collection of data features fαn , fβ

n , fγn , . . . is available. These

entities act simultaneously on the bridging entity. Both α and β can interact with

the components of Bn.

Termination and Equilibria

The phase is a decision point; it determines whether to set up the environment

for more emergence, or to allow the synthesis of KD to occur. Allowing synthesis

triggered by the occurrence of at least one of two events.

The expression (originally from Equation 3.1)

X (R (ρn+1) = R (ρ0)) (3.22)

determines whether the currently implied expression in the K-space form is likely

to refer to the same referent R as the data. As long as the estimated confidence


(in a reference state) is below a threshold, the system keeps looking for better sub-

descriptions until the confidence Xn+1 is sufficiently high or equilibrium is reached.

Equilibrium is when the system detects that it will not increase its confidence. A

sliding window of confidence estimates would suffice to notice whether the confidence

is appreciably improving.

Actual identification or construction of K-space can be left until synthesis is

triggered. B∆ associates configurations within Bn+1 with expressions in K-space.

If the content of Bn+1 is comprised solely of configurations already present in B∆

and it is easy to identify each configuration, then there can be a high confidence

in Bn+1. This confidence is denoted Xn+1. The identification of a configuration

is based on matching configurations in B∆ with configurations in Bn+1 and taking

tolerance into account. A brute force measure of every pair of configurations across

the two representations using tolerance to determine their compatibility could work,

but would have O(n2) computation. If B involves a natural dimensional separation

or is a representation whose elements are easily resolvable, this would reduce the

burden of brute force.

If Bn+1 is comprised of both recognizable and unrecognizable configurations.

The confidence Xn+1 will be based on the familiar configurations, and the absolute

richness and relative proportion of information familiar:unfamiliar.

3.3.2 Synthesis

The second stage involves fetching those description fragments, and composing them

into an alternative description in the K-space. This alternate description is denoted

as KD. KD is a reference to the same referent, R, that was originally referenced by

D.


Modeling Bn in K-space

Once Xn+1 ≥ threshold, K∆ can be synthesized out of the currently associated

or implied K-space expressions. These expressions correspond to the Bn+1 config-

urations that are familiar within tolerance and other expressions that need to be

inferred from configurations that are deemed unfamiliar. The association of familiar

elements is recorded in B∆, it is a mapping between B-space and K-space, though it

is not necessarily an item to item mapping. Inferred K-space expressions correspond

to the addition of new knowledge, to the system, about the current situation.

The familiar configurations within a region, specified by measure, are proxies for

the K-forms that will be combined in the making of new K-forms. Alternatively,

part of each familiar configuration and/or K-form, subject to measurement in the

B-space, can be used.

The finer granularity of B-space will not necessarily improve emergence from the

interactions. It might enable better construction of K-space expressions, or more

accurate association within familiar K-forms. If multiple associations are associated

with a B configuration, the X bestows a mandate upon the system to collate every

associated K-form. Though this might not be the most accurate description of

R, it is an acceptable description as determined by Xn+1. Multiple instances of

a configuration can suggest a strength of presence of concept. Similarly it could

indicate the strength or related confidence of the configuration. It is dependent

on how spatial, temporal or other directly measurable information is preserved or

transformed in the domain. This can be part of the calibration, and as such is not

a focus of this thesis. A more sophisticated system or system calibration should be

able to distinguish countable items from strength of membership.


Intangible Composition

The referents are intangible. The final stage of composition also involves the intan-

gible. The final tangible output, of a system employing the framework (Kfinal) is

a reference which indicates a referent, whose drift from the data referent has been

deemed acceptable by means of a confidence estimate.

Though the Kfinal in Figure 3.9 is notionally a reference to a situation, Kfinal

can be comprised of many elements, all of which are indicators of the referent.

The follower of the indicators is the recipient, who associates with the descriptive

elements presented, events from the recipient’s own experience. The framework

attempts to gather a collection of references that the recipients will put together

themselves. Ideally they attain the same referents as an expert, as when an expert

looks at the original situational data (or referents that an expert would consider a

plausible interpretation).

3.3.3 Summary

The use of the framework assumes a pre-existing repository of domain knowledge

(K0) in the same form as that of desirable description. It also assumes a resource of

emergence calibration, held in an augmented bridging entity (B∆). In Figure 3.11,

the bridging entity B0 is transformed into another bridging entity configuration, Bn.

RDRKfinal

D

K0 B0

Bn Kfinal

Figure 3.11: Reference Transformation


B0 is perturbed via progressive interaction with the data D. It is employed as an

intermediate level of reference, bridging between the situational data references (in

D) and the description references (in Kfinal). The last step of the transformation,

uses B∆ to map from the form Bn to the form Kfinal.

Kfinal is a combination of initial general domain knowledge and specific situ-

ational references. An attempt is made to preserve the same referent, RD, as is

indicated by the data D. Acceptability of transformation is based on confidence

estimates that referent drift is not too great. The confidence estimates are based

on framework associations in B∆, which were in turn based on calibration of expert

descriptions of other data.

The main limitation is that drift from the referent cannot be measured directly.

For there to be no drift, Kfinal = KD. KD is only potentially available for calibration

instances of situational data. Apart from an equality test on the entire situational

data, this cannot be verified. Even if it could be and is compared, it would be

simpler to then simply return the calibration description. Given than some drift is

expected, the focus is on reducing drift.

The emergence techniques, used for calibration, are also used for the interpre-

tation of new situational data. The framework assists in the emergence of tangible

references that will encourage intangible recipient understanding.

Chapter 4

Bridging Surfaces

Two slightly different representations of the domain knowledge are used. One version

is in the form deemed most accessible to the recipient. The other is used during

emergence and transformation. The knowledge representation (of the domain) exists

independently of framework transformation. However, because of this it might not

be suited to perturbation via interaction with data. The knowledge space domain

description is used as a first approximation for a situation description (for any

situation). There is a corresponding description, in the bridging space, known as

the initial bridging entity, B.

The use of bridging entities are a central aspect of the framework. They enable

interactability of knowledge elements and data elements. Their ability to be modified

allows data elements to indirectly interact with each other. Bridging entities allow

both calibration and interpretation to benefit from emergence without requiring

implementors to define the results of interaction.

Data that is specific to a situation, influences the perturbation of bridging

entities. A record of prior perturbation by calibration data enables interpretation

of an otherwise undescribed situation.

98

CHAPTER 4. BRIDGING SURFACES 99

This chapter shows how a perturbable bridging entity can be thought of as a

malleable surface. The metaphors of deforming and folding a surface correspond to

modifying a description and manipulating descriptive elements respectively.

The inspiration for folding as a bridging technique came from origami and protein

folding. In origami, paper is folded to represent real world referents (such as animals)

and mathematical objects (such as polyhedra). Traditionally, models start with a

single square of paper. Situational models always start from B0; the framework’s

parallel of the single piece of paper. The surface metaphor differs from the traditional

single square in that it isn’t considered initially flat. More on this in “Ramifications

of a Flat Surface” in section 4.1.2. Often origami models can be ‘recognized’ as

real world objects, even though they are abstractions. The origami model can

be seen as a collection of flaps, where each flap is part of the paper that can be

manipulated independently of the rest [Lan01]. The flaps collectively suggest the

real world object. The surface parallels of flaps are folds which are ideally, though

not necessarily, independent.

When protein molecules fold, the shape of the folding holds information that

is interpreted by the cell. The folding of the protein molecule is an emergent

phenomenon, where the interaction with the environment modifies the shape of

the protein molecule. In the framework, the bridging entity is the analog of the

genomic material and the data corresponds to the environment.

The surface is a substitute for the destination reference form, as it is the des-

tination form (the description) that is transformed rather than an instance of the

source reference form (the data). The bridging entity is modeled as a manipulatable

surface. While this is not the only way of modeling a bridging entity, it has the

benefit of providing a way of thinking about manipulating parts of a bridging entity.


The initial surface is initialized to the representation and conformance of initial

knowledge. The final surface shape corresponds to situational knowledge or inter-

pretation. The final description will be in terms of what is known: K0 in particular

and K-space in general.

4.1 Surface as Metaphor

4.1.1 Folds

A surface as a whole is denoted Φ.

Φ = [ϕ] (4.1)

In Equation 4.1, ϕ is an individual fold, and [ϕ] is a population of folds (in this

case composing the surface Φ). The typical curled braces, ‘{’ and ‘}’, of crisp set

notation are not used, as folds are allowed to overlap. No isolation of an individual

fold is attempted. The notion of folds, is an attempt to explain the progression of

overall emergence when surfaces are used as bridging entities. It is possible that

all uninterpreted folds might be dealt with as one unfamiliar fold. Surfaces are

metaphors for the results of interaction. The actual forms of the modified bridging

entities are not necessarily multi-dimensional geometric curves. However, the surface

metaphor aids in thinking about how to use the results.

A bridging entity Bi can also be seen as a surface Φi: a shorthand notation for

ΦBithe surface fashioned by the initial bridging entity Bi. Φi will be used when

talking of surfaces specifically, and Bi will be used when talking of bridging entities

in general.

ρi = [Di, Φi,K0,B∆] (4.2)


Equation 4.2 denotes the reference state after i iterations. A reference state

comprises the situational data Di (which includes information about what data

portions have already been involved in emergence invocations), the surface after i

invocations of emergence Φi, the knowledge source K0, and the store of experience

B∆.

Note that K0 and B∆ stay the same, as the knowledge source and system

experience are assumed to be immutable during reference transformation. As K0

and B∆ are held constant an abbreviated notation can be used simply signifying

ρi = [Di, Φi] (4.3)

Invocation of interaction is denoted by Ψ

ρi+1 = Ψ(ρi) (4.4)

In Equation 4.4, Ψ indicates that the constituents of ρi interact with each other

to realize the next reference state, ρi+1.

Expanding using equation 4.2:

ρi+1 = Ψ([Di, Φi,K0,B∆]) (4.5)

The surface which initially maps to the entire domain knowledge is modified by

the data in such a way that the final surface describes both the situation described

by the data, and also the rest of the domain not directly applicable to the situation.

The final surface composition includes the cumulative deformation caused by

interaction with the data. The data is an indication of the situational referent. The

final surface contains descriptions of both general domain and specific situational

phenomena. The final situational description is further interpreted and used by the


recipient in a way analogous to the recognition and use of a protein by a biological

cell.

Similar to an origami model being comprised of individual flaps, a surface con-

tains individual folds. These collectively suggest the real world object or notion

(first suggested by the data). Unlike origami, the folds in the surface need not be

physically separate in implementation. Though they are notionally resolvable (e.g.

as individual overlapping signal components can be separated by Fourier analysis).

A fold is a region resulting from interaction (in B-space), which corresponds to a

descriptive form (or forms) in K-space. That K-space form does not have to be

familiar (i.e. already observed in K0, or experienced in the calibrated B∆).

The crux is the triggering or matching of relevant folds from calibration, and the

determination of their meanings. If the fold can be ‘recognized’ as an instance of

the group or range of folds corresponding to a known phenomenon, the associated

K-form can be used as part of the overall form.

If the fold is not recognizable as indicative of a known phenomenon, either

a composite K-form is hypothesized or it is flagged as unfamiliar. It will not

always be possible to distinguish known, partially known and wholly unfamiliar.

All surface regions that do not correspond to familiar calibrated folds, contribute to

the descriptions of unfamiliar potential phenomena for the purposes of confidence

estimation. Estimation of the proportion of unfamiliar surface regions can be based

on the collective surface of familiar regions, and the collective surfaces of unfamiliar

regions. The latter is simpler to estimate than isolating individual concepts within

the unfamiliar. The confidence with which the framework interprets the surface is

dependent on the estimated proportion. Confidence can be simply used to order

the presentation of situational descriptions. The framework can advise the recipient


which candidate description possess high framework confidence. The termination

criteria would not necessarily have to limit the results presented to the recipient.

The results of calibration of surface folds are contained within the bridging entity

denoted as B∆. B∆ stores what was observed when calibration data modified

the bridging entity. The calibration data indicates situations, for which expert

constructed K-space descriptions also exist. The corpus used for calibration need

not have been built with the framework in mind, though the descriptions of data

referents will have needed to be described in K-space. There exist regions in B-space

that have uncertain meaning. Folds resulting from the situational deformation of the

knowledge source can occupy these regions. The iterative process (in Section 4.2.4)

facilitates improvement in confidence of interpretation. The final collection of folds

after iteration, will be mapped to a collection of K-forms. The meaning of the

collections of K-forms is left to the recipient of the framework.

A result formed from the interaction between data and knowledge (described in

a K-space) will occupy a space B-space, where the dimensionality of the B-space is

at least that of the K-space.

A simple metaphor for this is the increase from a single-dimension space. In

Figure 4.1(a) a line representation occupies a single dimension of a 2D K-space.

In Figure 4.1(b) a perturbed line representation occupies > 1 dimensions of an

n-dimensional B-space. The data deforms the descriptive reference from straight

to curved. The number of dimensions occupied increases from 1 to > 1. The

description curve is considered to be the B-space reference corresponding to the data.

References can be mapped from the less complex K-space to the more complex B-

space. There is an interest in keeping the B-space-form as close to the K-space-form

as possible, as some B-space-form elements might be presented raw to the recipient.

The associative map-back to K-space is not necessarily limited to mapping from the


K-space projection of the B-space perturbed description. The calibration held by

B∆ can account for the dimensional change from the knowledge space.

.

..

..

..

.

..

..

..

..

.

.

..

..

..

..

.

..

..

.

..

.......

.

.

......

.

........

.

.........

(a) (b)

Figure 4.1: Surface metaphor for a reference in a knowledge space

Extending this metaphor to more than two dimensions, it is possible for the

number of dimensions to grow further than the growth from the initial interaction.

The interpretation of the result set is restricted to dimensions that have been

calibrated. This might entail information loss. Those values in the calibrated

dimensions are modeled as a surface.

Individual folds of the surface, involve particular subsets of the resultant values.

Recognizing folding is a recognition of value (group/region) perturbation.

To articulate the situational knowledge, to the recipient, the framework will need

to map back from the final surface (in B-space) to an expression of knowledge (in K-

space). This will be covered in section 4.3. Before attaining the final surface, there

can be several intermediate surface states depending on how the data is interacted

with the domain surface.

The resultant surface Φ, from the interaction of D and B, is like a hyperplane

where the topography matters. The surface topography is the content; not the

separator (as would be the case for discriminators in neural networks and other

classifiers). “Curves” and “sub-curves”, representing concepts and sub-concepts

respectively, can be read from the surface. Distinguishable, localized regions of


curvature, or certain types of curve, are referred to as folds. A fold or collection of

folds will have ranges of curvature which correspond to meaning.

The general case is data interacting with a surface (bridging entity) to produce

another surface. The initial map from K0 to Φ0 only needs to be done once.

4.1.2 New Folds

Tolerance

Mathematical structures known as tolerance spaces formalize resemblance within a

certain error [Sos86]. Semantic mappings can be made fault tolerant [MY08]. Enti-

ties used for calibrating surfaces will have tolerances and uncertainties for familiar

folds. The symbol ϕ is used to denote a fold within a surface. This discussion is

limited to two dimensional curves for ease of diagramming, though ϕs could exist

in n-dimensional surfaces. If a new fold is containable within the tolerances for

a particular calibrated fold, that fold is hypothesized as referencing part of the

situational phenomena. For most folds, three regions will exist: “within tolerance”,

“within uncertainty”, and “unknown”. The unknown region can contain other ϕs.

Within tolerance is a region which corresponds to a range of folding experienced

with prior data-meaning association. “Within uncertainty”, is a region whose size

is inversely proportional to the richness of the knowledge source’s prior associations.

Folds within this region would ‘currently unrecognized’. Further perturbation of the

surface (via subsequent iteration) could shift the folds either ‘within tolerance’ or

into the ‘unknown’ regions. Occupancy of the ‘unknown’ region possibly signifies

new knowledge detection. However, this would also occur for noisy or meaningless

data.

In Figure 4.2 ϕ0 is the pre-fold surface section corresponding to a notion. ϕcalib1

and ϕcalib2 are extremes of the various deformations corresponding to the view.


tolerance

ϕcalib1

ϕ0

ϕcalib2

Figure 4.2: Several measurements within tolerance, are associated with the samemeaning

��

�� uncertainty

band

��

��

(same meaning)within tolerance

��

��

��

��

��

��

��

��

��

��

��

��

Figure 4.3: some measurements have uncertain meaning inversely proportional todata richness

Uncertainty Bands

Finer fold localities improve the chances of distinguishing topics otherwise at risk to

cross-topic interference. Though there will also be a greater number of uncertainty

regions. However, to make use of this information, it would need to either be

separable by topic, or clear that that there are no other topics involved.

If there are two concepts referenced in the data, for both to emerge, there need

to be folds within tolerance for each of them. The data indicating each concept (and

possibly overlapping with the other data) should modify the bridging entity such


that the collection of folds are resolvable within tolerance for appropriate mapping

back to the concept indication in K-space. There is always a danger of cross-

concept interference in B-space causing otherwise properly shaped folds to drift into

uncertainty bands or beyond. Shifts in these bands or regions, will lead to changes

in the confidence estimates that will be associated with description references.

New indication is learnt in terms of familiar indication, and might be part of

the familiar indication. Uncertainty bands, while associated with individual fold

regions, will collectively remove regions of the fold-space from consideration either

for detection of familiar situations or determination of unfamiliar situations. Their

benefit is the ability to state that the data refers to a different possibly new concept.

The uncertainty bands have to be calculated or estimated, and assigned to parts

of the (possibly infinite) surface Φ0. The proportion of the fold-space removed due to

uncertainty will be proportional to the richness of the data available for calibration.

B∆ contains augmentation and calibration of B0 by many situations referenced

by separate collections of situational data.

Calibration shows what collection of measurements denotes a fold in the surface,

and how this is interpreted in the knowledge representations of K-space. A deformed

surface is comprised of a collection of folds. A candidate structure for a surface

cannot be a crisp partitioning of a knowledge space K; there needs to be scope for

new deformation and uncertainty bands.

If a fold exists outside the tolerances of all existing folds and is also not within

uncertainty regions, the existence of a new aspect of an existing situation or an

unfamiliar event is hypothesized. No map from the B-space fold to a K-space form

would be available. Though the fold might be interpretable by the recipient in a raw

B-space-form shorn of non-K-space elements. The confidence in the entire reference

is based on the familiar.


If a fold exists in the uncertainty region of one known fold, the hypothesis is

that it is either the K-space-form corresponding to that fold, an unfamiliar event,

or an ambiguous event. Disambiguation can wait for verification with other data.

Multiple hypotheses can be generated from the same surface, if multiple calibrated

folds match the emergent bridging surface’s folds. Verification and augmentation

occur during further iteration of the invocation of emergence.

Ramifications of a Flat Surface

While the choice of initial surface shape can be somewhat arbitrary, two shapes bring

themselves to our attention: a flat surface (analogous to a flat square piece of paper

prior to paper folding) or a surface which is a representation of the domain knowledge

K0. The choice would have been made when the meanings of deformations of

surfaces were calibrated. For the calibrations to be most meaningful, as much of

the original conditions as possible need to be retained. It is also assumed that this

is the best way to retain the same context as calibration. The system has been

calibrated such that deformations of the surface will correspond to a state of the K

space, a description. In the case of a new situation, the description in K is not to a

representational instance that had existed previously.

Use of a flat surface requires that the emerged result only describes what was

in the situation as indicated by D. It indicates nothing about anything else in the

domain. The assumption is that what is not disagreed with is also still ‘known’ but

not specific to the data.

Noise

Noise is different to cross-topic interference, being a degradation in the quality of

the data, rather than a confusion regarding interpretation of the data.


Noise is mentioned further, in the situations. However, it is worth noting that

surfaces work less well with noisy data. Everything interacts, including the noise.

Both richness of data (that contains the necessary specifics) and noise, will perturb

the deformation of the surface.

Calibration is easier if transformations caused by that context’s non-noise is

favored over transformations caused by the noise.

Noise exists in the form of statistical distributions and errors. As with stochastic

phenomena, statistical noise can be anticipated in bulk but not in detail. Some

statistical noise will be able to be removed from the data, in data cleaning, before

it reaches the framework.

Data errors can be partially overcome by looking at supporting data for verifi-

cation. If each data instance, d, interacts with an unchanged B0, information the

information will need to be combined later. Alternatively each data instance could

interact with a Bi, where Bi is a steadily augmented and modified version of B0. In

the latter case there would be a gradual lessening of confidence due to calibration of

interacting Dcalib with B0, where Dcalib is the entire situational data used in prior

calibration.

4.2 Fitting the Framework

A situation, which needs to be described by a reference in K-space, is posited.

The situation is currently described by the data, D. There exists a source of

domain knowledge K0. System experience is represented as a calibrated recording

of information (from prior invocations of the framework) B∆. There exists an initial

bridging surface Φ0, which is a derived representation of K0.

Emergence occurs in the system, when invocation of the framework causes indi-

rect interaction of the constituents of D with each other via Φ0. In the final state,


there is a situation indicated by Kfinal as mapped from Φn, where n is the number of

iterated invocations of the emergence interaction. Some sub-forms of Kfinal will be

associated with familiar sub-forms of K0; others will be unfamiliar forms in K-space.

4.2.1 Surfaces as constituents of Reference States

Surface states are discussed: first in terms of final state requirements, then the initial

state by which all situations are approximated, and then by the intermediate states

that are the progression from initial to final. It is this transformation that is the

focus of the framework.

Final State

A final reference state

ρn = [Dn, Φn,K0,B∆] (4.6)

is reached. The surface Φn which is also a reference, is the nth reference to a

situational referent. if either the confidence in the situation description exceeds a

threshold

X (R (Φn) = R (Φ0)) > Xthresh (4.7)

or

ρn ≈ ρn+1 (4.8)

Equation 4.8 can be re-written as

ρn ≈ Ψ(ρn) (4.9)


In other words, termination can also occur, if the invocation of emergence fails

to appreciably modify the reference state.

Ψ(ρn) signifies invocation of emergence that involves the constituents of ρn.

Ψ(ρn) → Φn Υ Dn. K0 is the domain knowledge in an accessible form. B∆ is

the store of calibration experience.

If there is no more information to extract, the interaction of Dn with Φn should

not create a surface Φn+1 whose associated knowledge form is appreciably different

to that of Φn.

Dn is not just the totality of data available after iteration n, but also represen-

tative of the data portions dj that have been interacted with the bridging entity.

The portions can be determined by natural partitioning (if any). The nature of the

natural partitioning might also suggest a strategy for progressively interacting the

data portions dj with Φn.

Initial State

ρ0 = [d0, Φ0] (4.10)

For Φn to retain the information the framework gleaned from the previous n

steps, Φn−1 needs to reference the situational events that might not be indicated by

data portion dn−1. Φ0 is a surface that indicates aspects of the domain also indicated

by K0. The aspects unchanged from K0 will be either ‘untouched’, ‘highlighted’ or

not contradicted by occurring perturbations. Assumptions are made with regard to

normality represented in K0 unless contra-indicators are present. Φ0 should not be

a flat or otherwise non-associated surface.

This assumes that the errors in context due to perturbing already deformed

surfaces (in the chain ρ0, . . . , ρn, ρn+1) are small compared to the perturbation of

the description by the data.


Referent Drift

Though the framework focuses on reference transformation, the critical subject,

under discussion in the thesis, is the maintenance of an acceptable data referent.

The data referent can be seen as an interpretation of the situational data, had the

interpretation been made by an expert whose K-space descriptions were used during

calibration. Referent drift is the degradation of acceptability of of the interpretation

that will be made by the recipient upon receival of the final K-space description from

emergence.

The essential problem in estimating confidence is that no equality or similarity

test can be directly performed on the referents which are intangible in an unmea-

surable space. The referents exist primarily in the minds of the experts and/or

the recipients. An attempt is made by the framework to produce a K-form that

encourages a recipient referent that would be acceptable to an expert who views the

original data reference.

Referent drift is the gradual displacement of the referent in the possibly un-

measurable referent space. This gradual displacement can be increased by the very

iteration that seeks to further modify surfaces toward more acceptable interpreta-

tion, due to increasing invalidity of previous calibration. The framework transforms

references while keeping track of estimates of the change to the referent. Not all

reference modifications will result in referent drift.

Kfinal is comprised of the last references to the situation, which are then pre-

sented to the recipient. The recipient can then further interpret the situation,

using the collection of descriptions in Kfinal, possibly in conjunction with other

information from other sources.


Intermediate States

The transition states are

ρi−1 = [Di−1, Φi−1] (4.11)

and (4.12)

ρi = [Di, Φi] (4.13)

Upon termination of iteration the accumulated error will have been affected by

the amount of progressive deviation of the surfaces Φi from the initial surface Φ0

and the number of iterations.

There is an increased confidence due to invocation; there is a decreased confi-

dence due to drift. If the decrease occurs faster than the increase, it is not worth

progressing as the overall confidence would be less than the point at which the

decrease exceeded the increase in magnitude. It might be worth backtracking to

invoke the emergence chain multiply to verify hypotheses rather than allow the drift

to get out of control. The backtracking would be from Φn to Φm where m < n. As

the drift is difficult to estimate, there could be a heuristic, applied to confidence, for

gauging when to backtrack.

Ideally, it should be detectable whether the rate of change of confidence is due

to drift rather than lack of knowledge. A very new phenomenon would also cause a

low confidence.

All intermediate states occur as surfaces in the bridging space. Though all

bridging surface states potentially correspond to K form representations which have

some similarity to KD. Notionally they all reference the data referent r[D], though

some drift is expected for all referents R (ρn). Given the possible augmentation of


B-space folds with respect to K-space elements, the B-space references will only be

partially interpretable by the recipient.

Confidence: Referent Drift

Referent drift is estimated by the confidence in the final surface Φfinal. This in turn

is estimated from the proportions of familiar surface regions, and from accumulated

folds and their individual confidences.

Confidence in an interpretation is greatest for situational data whose effect on

Φ0 causes the surface to adopt (within tolerance) the form of folds that emerged

during calibration. The confidence XΦican be based on the proportion of tolerance

matching folds to ‘uncertain’ or ‘unfamiliar’ folds in the B-space. The only folds

in Φ0 that have tolerances, will be those that also occur in B∆. X will be based

on the proportion of of ‘certain’, ‘uncertain’ and ‘unfamiliar’ folds. The ‘area’ of

unemphasized existing folds is difficult to estimate. Using only part of Φi to make

it easier to estimate proportions, runs the risk of missing newly emerged description

of previously implicit data.

4.2.2 Interaction

Interaction occurs between the data and the bridging entity. From Section 3.3.1

D Υ B → B (4.14)

This says that the results, of the interaction between data entities and the

bridging space (B-space) entities, are also entities in B-space.

Equation 4.15 can denote the interaction of the initial bridging entity (modeled

as a surface) with the data:


D Υ Φ0 → Φ1 (4.15)

During interaction, a data portion dn interacts with a surface Φn to yield a

perturbed surface Φn+1. This more general case is discussed further in section 4.2.4.

In the first invocation (see equation 4.16), interaction with the data causes the

surface Φ0 to become deformed creating the modified surface Φ1. The components

of the data combine with the components of Φ0. The results of those combinations

are the measurements that will be interpreted as a surface, as they would have

been during calibration. The calibration process will have ensured that the result

characteristics had surface-like qualities.

d0 Υ Φ0 → Φ1 (4.16)

The portion of D that is d0 is comprised of some of the features from the input.

Any (if not all) of the features can interact with all bridging entity elements, which

are not necessarily the same as folds (which are a theoretical bookkeeping of the

surface with respect to semantic elements). The initial bridging entity elements are

more likely to have similarities with a population of knowledge elements [k.

Φ1 = Φ0 + ǫmeasure (4.17)

In equation 4.17, Φ0 is the bridging surface before interaction; Φ1 is the bridging

surface after interaction. ǫmeasure is the measurable perturbation. Some of the

perturbation will correspond to the data, and some to drift.

Φi+1 = Φi + [ǫdi , ǫdrift] (4.18)


To resolve ǫdi and ǫdrift would be convenient, as the tolerances could be gauged

against just Φi + ǫdi . However this is not necessarily possible.

One consequence of prior calibration is that the surface perturbation from a

particular surface will be interpreted according to the descriptions of the calibration

data. Note that K0 would not have been built from the corpus data. It was a

description of the domain using the K-form. The K-form would not have been

machine generated as a result of the interactions. The first K-form K0 is the basis

of the first B-form B0, to enable easier mapping back. There is an issue whether a

resultant surface Φi is sufficiently close to a valid K-form to be interpretable (without

mapping back) by a recipient. This is more important if there does not exist a way

to map from a fold to K-space. The final K-form results from the perturbation + a

map-back (if requested). Calibration, as well as providing tolerance information for

folds, holds mapping information.

K0(conversion) → Φ0(transform) → Φn(mapback) → Kfinal (4.19)

The Φ1 form is not necessarily a K-form that reflects the referent indicated by

the data D or even the data portion d0. A different feature extractor would result

in D′

and d′

0 which when interacted with Φ0 would yield Φ′

1.

d′

0 Υ Φ0 → Φ′

1 (4.20)

The calibration information in B∆ would be different. The mapping back would

be using different values.

ǫdrift refers to the drift due to interaction with a bridging entity other than Φ0. ǫ

should be within the tolerances for Φ0 Υ d0, or those specified in B∆. ǫ represents


the actual drift which is not measurably known; it will be estimated, or allowed for

in techniques which implement the framework.

Drift is the deviation from the likely results of interaction of the data with Φ0

(the surface used in calibration). The further the deviation, in form of the B-space

folds from the folds of Φ0, the greater the potential error. This is almost a necessary

introduced error, since the possible lack of measure in K-space would make deviation

in K-space more difficult to address. The situation (as initially described by the

data) will contain normal or familiar events that are expected to translate to normal

or familiar expressions in B-space and K-space. These folds are “domain folds”. The

created domain folds are expected to lie within tolerance. Interpretation of non-

domain folds is expected to lessen confidence that the referent has been maintained.

Though some non-domain folds will result correspond to significances of a situation,

confidence in their interpretation will affect their acceptability. Partitioning unfa-

miliar surface regions, or other attempts at identifying individual unfamiliar folds,

might involve error due to non-linear loss.

Ideally, the same data that interacted before, should have no further effect on

the surface if interacted again (see equation 4.21). Alternatively it could be a

reinforcement of the same notion. The notion of reinforcement or emphasis of a

K-form ties in with domain knowledge descriptions that are particularly present

or significant in the situation. For example, there can be indications that there

is a leopard in a jungle, but that notion would likely be represented in a wildlife

knowledge source. For the recipient to be made aware of the likely presence of

a leopard, attention should be drawn to a K-form (representing a leopard) which

already exists in K0 and Φ0.

d0 Υ Φ1 → [Φ∗

1] (4.21)


where

Φ∗

1 = Φ1 + ǫemphasis (4.22)

where ǫemphasis is interpreted as emphasis of something already assumed. This

can be part of the information held in B∆. If the expert descriptions used in

calibration were biased toward the interesting over the normal, it could be that

emphasizing information is less available than possible in the map back to K-space.

dfamiliar Υ Φi → [Φi, ǫemphasis] (4.23)

Unfamiliar data would result in a surface that would be different to the interacted

surface, but hopefully with the perturbation only being interpreted as an emphasized

K0 form if that unfamiliar data references the same referent as the existing K-space

form.

dunfamiliar Υ Φi → Φj (4.24)

If the next data portion (if another exists) is also interacted with Φ0 (see equa-

tion 4.25) to avoid the drift due to calibration inapplicabilty, the problem of combin-

ing information is raised. Φ1 would have gained references specific to d0; Φ′

2 would

have gained references specific to d1.

d1 Υ Φ0 → Φ∗

2 (4.25)


Measurement and Tolerances

The results of interaction on first glance would seem to be amorphous clouds of

values. The value clouds are seen as surfaces. The difference between one cloud and

the next, is seen as the perturbation of the surface.

The calibrated folds and associated tolerances are held in B∆.

A surface provides a measurable space, where the measure is defined by prior

calibration. The calibration will have involved several situations; each described by

both a data instance, Dc and an accepted description, Kc.

There is also an initial surface, B0, which is an abstraction of K0. It will have

been present prior to calibration of B∆. Each Dc perturbs B0 causing emergence of

a collection of results. These results are modeled as another abstract surface, where

the differences are collectively considered a deformation of the surface.

The aim of the calibration is to associate deformations of surfaces or local

deformation regions, with descriptions or sub-descriptions respectively. Similar sub-

descriptions will have an associated collection of local surface regions (folds), and

tolerances which specify known variations in the folds.

The reason that Dc interacts with B0, (the abstraction of K0) rather than K0

itself, is that the same kind of interaction needs to occur in both calibration and

each stage of the transformation chain (See Equation 3.7).

In order for previous deformations in the chain to be remembered, each successive

Dn has to interact with the corresponding Bn.

When new data is interacted with a surface, it initially interacts with Φ0, not B∆.

For each fold generated, if it falls within tolerance for a composite fold group in B∆,

an associated descriptive fragment will be provided to the composite description.

There might be drift even if a data set Dcalib, used in calibration, is used as situ-

ational data D. The fold tolerances would be based on all calibration instances. All


calibration data instances, when interacted, should result in folds within tolerance.

However, the more inclusive the tolerances are of all data calibrated, the higher

the chance of matching of multiple hypotheses. Though this in turn is affected by

fineness or specificity of tolerances. The latter is slightly better if at least one of

the multiple hypotheses is deemed of acceptable confidence. If only one hypothesis

is maintained, the situational data would be interpreted similarly to the calibration

data, if the data is partitioned and seeded the same. However, depending on the

domain, a consistent seeding of partitioned data might not be possible. See also the

section (4.2.4) on multiple hypotheses from many seeds.

Limitations exist for the interpretation of folds which lie outside all tolerances of

B∆. Several competing interpretations can exist, if some folds individually satisfy

the tolerances for multiple folds in B∆. Each matching fold will associate with

descriptive elements in K-space, meaning that potentially multiple K-space-forms

exist. These are not multiple classifications so much as multiple attempts to describe

the data referent.

A related issue is that of the authority conferred upon descriptions during prior

calibration. Depending on how experts have described the content of data corpuses,

the significance and tolerances of a particular fold could have varying uncertainty

related to the granularity of description. A description or description fragment will

be authoritative for as a whole for the extent or sub-region of the data that it is

associated with. How decomposed calibration data is associated with fragmented

expert description, is an issue for the technique of calibration. However, it does raise

the possibility that multiple interpretations will exist for any given fold.

Confidence takes into consideration analogs of the classification concepts of false

positives and false negatives. A false positive is a description that purports to

refer to the same referent as the data. A false negative is something flagged as


either outside the descriptive capability of the knowledge source or a an acceptable

description that has low confidence. A false negative, at the level of interaction, is

an interaction that does not result in a recognition of significance. The result of

such interactions can be null if they either do not interact or they interact and cause

no distortion linked to association, or they cause a distortion that is misleading.

4.2.3 Fold Populations

A given fold, ϕi, might correspond to a sub-referent of a situational referent. Though,

there might not be any construct in B∆ that provides a map from ϕi to K-space.

However ϕi (ostensibly in B-space) might be interpretable by the recipient even

though it is not mappable back to K-space. It might be interpretable even if it

is indistinguishable from the rest of the surface Φ. Entities in B-space principally

differ from K-space in augmentations that enable them to be folded (perturbed).

Though the augmentations introduces noise, regions of the surface might still be

decipherable by the recipient. A (possibly disjoint) region of a surface can contain

a population of folds.

Fold populations exist in both Φ0 and B∆. Φ0, like K0, contains possibilities;

references to phenomena that might exist in any given situation. B∆ in a sense

contains definitions whose ‘terms’ are in the final surface, and whose explanations

are in K-space. KD notionally contains actualities (ideal references to plausible

data referents); Kfinal contains guesses in K-space for those actualities. An ideal or

acceptable Kfinal. There can be many possible acceptable KD. If only one KD is

applicable, it will be whichever one is closest by significance to the hypothesis that

emerges. KD can contain both familiar and unfamiliar elements.

A situation is posited in which there exist four plausible phenomena. The data

to an expert would suggest four phenomena. The ideal knowledge form KD, which


theoretically refers to the same referent as D, would have at least four subsidiary

forms [kα,kβ,kγ ,kδ]. kα and kβ are familiar and known in K0. kγ is familiar and

unknown in K0, but known in B∆. kδ is a unfamiliar phenomena; unknown in both

K0 and B∆.

Assuming a 1:1 association of forms in Kfinal with folds in Φn, the ideal fold

population of Φn could be [ϕα, ϕβ, ϕγ, ϕδ]. More practically there would be an

extra fold ϕω that corresponds to all parts of the surface that do not correspond to

a K-form. These are known as remaindered folds.

The folds should be sought in the whole resultant surface, rather than just

the measurable perturbation. To merely measure the changes, would mean that

modifications due to earlier interactions would be ignored, possibly losing important

information.

Φi+1 = [ϕα, ϕβ, ϕγ, ϕδ, . . . , ϕω] (4.26)

allows persistence of interpretations from earlier iterations., as the folds would

reflect the combined perturbation due to all iterations.

Depending on the confidence estimates in Φn, some of the folds might be merged

with ϕω. This is more likely to occur when there are unfamiliar folds, seeing ϕδ

being indistinguishable from ϕω, say ϕδω. There could also be one or more false

positive folds, say ϕǫ, that correspond to forms in K-space that are misleading.

ϕδomega would be seen in terms of likely small confidence estimate as referencing a

possibly new phenomenon.

If ϕα, ϕβ, ϕγ and ϕδ are resolvable, Kfinal, the actual mapped-back K-form,

should have kα, kβ and kγ . Ideally, kδ would also exist in Kfinal - interpretable b

the recipient though not mapped using B∆. This could eventuate if, say, ϕω as is,

or perhaps with some attempt at cleaning its form.


There are likely to be more subsidiary forms than just kα,kβ ,kγ and kδ, which

are the same as other domain knowledge forms in K0 having been unchanged by

the emergence framework. It is domain dependent whether unchanged domain

knowledge is explicitly present in Kfinal or just assumed to hold in the absence

of contradiction.

When Φ0 interacts with the data, it generates forms not representable in K-space.

In the metaphor of the surface, points or regions from K-space are dragged to other

parts of B-space. In subsequent transformations, the surface can further expand

into B-space beyond what it occupied and expanded into before. The K-space is a

more constrained space than the B-space. Associative interpretations are restricted

to those dimensions of B-space experienced during calibration and recorded in B∆.

The movement of a point on the surface from its Φn state (in B-space) to its

Φn+1 position (also in B-space) is considered to be the distortion of that point. The

distortion of the point is part of the perturbation of the surface. A local collection of

distorted points is a fold; the collective distortions of all points is the perturbation

of the surface.

B∆ holds either the meaning of that distortion (or collection of distortions) or

the meaning of a particular topography of the surface. The latter is more straight-

forward to infer from, as only the shape itself needs to be measured, rather than

both measuring Φn+1 and calculating deviation from Φn.

The calibration of deformation, is with respect to Φ0, as that is the surface with

respect to which the B∆ points were calibrated. The tolerances refer to distortion

from Φ0. Meaningful fold of a calibration surface will have tolerances associated

with it.

Φn+1 is a collection of values (e.g. scalar, structured, sensory) that are inter-

preted with respect to the representation of B∆. The surface Φn+1, which is the


representation of the bridging entity, occupies a measurable space. The new surface

is considered a hyperplane, whose constituent points or parts can be measured and

moved in calibrated distortions, with respect to another surface. These distortions

were determined and calibrated after interaction of training data with Φ0. Toler-

ances, specified against those calibrated distortions, define regions in B-space.

If several folds exist, some number n of them will be situation specific folds;

some other number m will correspond to existing folds in Φ0. The surface originally

occupies a configuration in B-space corresponding to descriptions in K0. The

perturbed surface will match some folds with B∆ tolerances, some will stay as they

were initially, others will be ‘emphasized’, and yet others will be unfamiliar.

4.2.4 Iteration

Data Portions and Data Partitioning

The entirety of the data available from Dn in most cases will be not different to D0.

Dn can be different if new data is discovered after an invocation of transformation.

What actually interacts, dn, can be different from dn−1 if there is data available

that wasn’t used before. There can be unused data now available if the data was

partitioned before and/or new data has been discovered.

Φ0 Φ1 Φ2. . . Φn

d0 d1. . . dn−1

Figure 4.4: Interaction of data portions during iteration

In figure 4.4 the situational data, D is available in possibly overlapping portions,d0,

d1, ... , dn−1. The subscripts indicate the sequence in which the data portions are


interacted rather than a particular natural sequence. Emergence from other data

subitems either augment or verify/refute the emergence from their neighbors. If

multiple hypotheses are independently tested, intersection and/or overlap of mapped

K-forms are sought for verification. Independently achieved fold forms can reinforce

each other if sufficiently similar when mapped to their respective knowledge forms.

This however is unlikely unless particular data can be chosen specifically to test

earlier results. Otherwise arbitrary data can be independently chosen to interact

with Φ0 (or Φn−1, if Φn−1 was the prior interaction) in the hope of getting verification.

Modification of Φ0 using a second group of data, will reduce the error due to drift,

as Φ0 is just a copy of K0. The interaction of Φ1 with data incurs a drift from

calibration error as the calibration was performed using the interaction of Φ0 with

data.

To interact all data subitems (and all combinations of data subitems) with Φ0

independently would probably be expensive if not intractable. And all refuting

evidence would have to be considered; not only supporting evidence.

The Pros and Cons of Reduction

The potential non-linearity of interaction raises the issue of deviation of the complete

surface or collective deviations from individual folds. Reduction to individual folds

can decrease the errors due to mapping, by isolating the mapping of folds due to

unchanged familiar phenomena, from the mapping of folds due new phenomena.

Conversely, reduction of data can potentially separate non-linearly combining fea-

tures. The technique has to consider non-linearity even if it does not directly model

it.

Non-linear effects require data items to be interacted at once. More formally

(Ψ(d1,d2)) Υ Φn might result in a different surface Φfinal than either d1 Υ (d2


Υ Φn) or d2 Υ (d1 Υ Φn). The latter two might also be different from each other.

The associative rule is not guaranteed to hold.

KΦiis the K form that would be mapped back from the bridging surface Φi.

The symbol ∼ is used to signify sufficient similarity (what an expert would say

is a reasonable description of the data).

KΦi∼ KΦj

(4.27)

indicates sufficient similarity in K forms after both Φi and Φj have been mapped

back. However, consider the following:

Φi ∼ Φj (4.28)

This would be potentially misleading as the calibration handles both association

with K forms and perturbation of Φ0, but not similarity between emerged bridging

surfaces.

If d1 Υ (d2 Υ Φn) 6∼ d2 Υ (d1 Υ Φn), then the initial selection of data becomes

significant. There are two basic alternatives: symmetry and asymmetry.

d1 Υ (d2 Υ Φn)∼d2 Υ (d1 Υ Φn) (4.29)

d1 Υ (d2 Υ Φn) 6∼d2 Υ (d1 Υ Φn) (4.30)

If neither all data nor naturally occurring smaller data amounts are made to

interact with every fold, the remaining option is arbitrary or heuristic reduction to

sub-selections of either data or surface. As soon as arbitrary selection is introduced

to reduce computational complexity, there is a risk of partitioning a body of data that


should not be divided. Heuristic selections of fold-localities limit the representation

by anticipation. Furthermore, the boundaries of fold-localities are not necessarily

known.

If the data is added separately, it makes the assumption that either the combined

data effect is linear or the discrepancy is negligible, or able to be adjusted for.

Subset sampling (which separates unrelated phenomena, at the cost of splitting

related phenomena) can reduce the effective noise caused by unrelated phenomena

(concepts); a kind-of aggregate or cross noise which washes out separate phenomena.

However, this also loses data-wide cues. For example, in image data, subtleties such

as color gradient and softness are lost. At an extreme, impressionist paintings would

just be collections of dots.

This technique is better suited to data which is already partitioned; either

through natural segmentation as with tomography, or by requesting new data. The

data portions are defined by pre-existing boundaries rather than arbitrary decisions.

Symmetry and Asymmetry

Symmetry (Equation 4.29) needs to scale to all permutations of a transformation

chain:

dn Υ (. . . (d1 Υ (d0 Υ Φn))) (4.31)

For data that does have symmetry with respect to interaction, there is still an

error due to loss of non-linearity. However, symmetry is the best situation that can

be hoped for. Choice of domain of situations will still have to consider non-linearity,

and techniques for lessening the error due to it.

In order to reduce the error due to asymmetry (Equation 4.30) permutation

needs to be lessened if not minimized. The cumulative error due to Φi 6∼ Φj needs


to be lessened. Use of the framework, with asymmetric data interaction, will need

to be configured to approach the properties of symmetric interaction.

If a natural data portion sequence is not available after natural reduction of

D, varying interaction sequences might result in different hypotheses. This is

not unlike an expert looking at a situations from different angles and drawing

different conclusions. Though a ‘best’ interpretation might be possible after a more

comprehensive consideration.

Termination: threshold and equilibria

Transformation is invoked to take the reference state ρn to a new reference state

ρn+1. This was introduced in Section 3.2.1. In that section Equation 3.3 had Ψ

representing the invocation of transformation for a single iteration.

ρn = Ψ(ρn−1) (4.32)

The iteration either causes convergence to a reference state with acceptable

confidence, or results in references of insufficient confidence. The iteration process

modifies the bridging entity (the surface). The final bridging form is mapped to the

final knowledge form upon termination of iteration.

If the confidence is sufficiently high that the referent indicated by the nth refer-

ence state is the same as the referent indicated by the initial reference state.

X (R (ρn) = R (ρ0)) > Xthreshold (4.33)

then further transformation of the reference state ρfinal should not change the

references appreciably


ρn = ρ(ρn−1)Ψ(ρfinal) ≈ ρfinal (4.34)

ρn = ρ(ρn−1)Ψ(ρfinal) ≈ ρfinal (4.35)

Any subsequent knowledge representation should be very similar to the final

accepted knowledge representation.

Kfinal+1 ≈ Kfinal (4.36)

The decision of when to terminate iteration, is determined by the change (or lack

thereof) of the confidences associated with the hypotheses (population of possibly

acceptable surfaces) generated during iteration.

Termination of iteration occurs when either no measurable increase in confi-

dence is achieved over a number of transforming steps, or a suitable level has been

reached. A new reference is considered satisfactorily emerged if the confidence

level is acceptable. Setting a confidence threshold value will reflect the trade off

between computational tractability and sufficient emergence. The tractability issue

is to do with the time taken for emergence, in the absence of significant parallel

processing. Also, an increase in the amount of interacting data and an increase in

the interacting knowledge elements, increases the probability of false positives (that

folds will coincidentally match).

The framework transform is applied in a similar way for both calibration and

interpretation. They differ in that the former possesses associated descriptions, and

the latter generates them.

Unless a threshold value is arbitrarily given to the system during calibration,

the calibration iterations will continue until equilibria or a identifiable increasing


trend in referent drift is detected. The problem of manual interference with the

termination is again that of implicit information. Even the expert might not be

able to nominate where the salient indicators cease in the data.

As the calibration data come with associated expert description (in K-form,

there would have been a different purpose to the iteration during calibration. The

aim would have been to make sure that the map-back from the Kcalibfinal match the

K-forms [ki] of KcalibD

. There is a choice (made by the framework itself) of reference

state (or stage) to map-back.

The threshold is a way to stem unnecessary processing. Equilibrium is a more

natural solution. Albeit one that leaves open the possibility of many more concurrent

marginal hypotheses. The thresholding could be used both as a hurdle and as a finish

line, depending on the eccentricities of the domain.

Hypothesis pruning

Prior to subsequent invocation of emergence, a non-singular hypothesis population

can be pruned, by removing the hypotheses whose confidences are sufficiently low

as to be negligible compared to other hypotheses or a threshold value.

Multiple hypotheses reflect the spread of interpretation possible for a fixed

collection of data D given a particular Φ0. They are caused by multiple concurrent

seed data (different choices for d0) or by surface folds matching (by tolerance)

multiple folds from B∆. A more esoteric cause of multiple hypotheses would be

ϕω being split into multiple new folds (as with the more recent groupings of the

great sea beast constellation mentioned in the first chapter).

It will up to the implementor of the framework whether multiple hypotheses

remain during the mapping-back. This might simply require ignoring all but the

surface with the highest confidence value.


Confidence: knowledge loss and gradual/cumulative decay

Calibration would only have been based on surfaces that are initialized to Φ0.

Iteration could have terminated after the first invocation of interaction:

dcalib0 Υ Φcalib

0 → Φcalib1 (4.37)

Or it could have terminated after some detected point of equilibrium. This is

not straight-forward as confidences and tolerances evolve during calibration runs

of the framework. The criterion could the first downward trend of confidence, or

determined by the point of occurrence of the highest confidence level in an extended

iteration run.

dcalib0 Υ Φcalib

0 → Φcalib1 → . . . → Φcalib

final (4.38)

Mapping back occurs at termination of iteration for both interpretation and

calibration. The same criteria for termination is used in interpretation as well as

calibration. Knowledge loss, in the sense of referent drift, can occur after each

invocation of interaction. Not all referent drift will be knowledge loss as part of the

iterative emergence will interact more significant data (both implicit and explicit)

that will take the indicated referent closer to the referent(s) of the overall data

R (D).

4.3 B-space to and from K-space

Emergence and its benefits are seen in the framework via the modified bridging

entity characterized as a surface. When perturbations cease, the B-space surface


is mapped back to the K-space, for presentation in a form understandable by the

recipient.

The form, K, for a domain, need not lend itself to convenient measurement.

The bridging entity provides measurability, and also association with K-space. The

measurability stems from comparison of the emergent folds or perturbations of folds

with the folds and tolerances in B∆.

There are three related knowledge sources: K0, Φ0 and B∆. K0, the domain

knowledge, is fully representable in K-space. Φ0, the initial bridging entity, is

represented in B-space. B∆ requires knowledge of both, K-space and B-space,

as it needs to assist in mapping from Φfinal (the final configuration of the surface

after iteration terminates) to Kfinal (the situational description that is presented to

the recipient).

K0 reflects the frame of reference of the recipient. It exists independently of

the framework and the transformation of indication of new situations. However,

because of this, it might not be suited to manipulation via interaction with data.

K0 contains descriptions of the domain itself. The surface Φ0 is kept as similar in

structure to K0 as possible, as not all emergent surfaces will be mappable back to

K0, and might be viewed raw.

The reason for Φ0 existing, is to be able to interact with data. There is a map

from K0 to Φ0. There is iteration from Φ0 to Φn. Finally there is an associative

mapping from Φn back to K-space as part of the synthesis of Kfinal.

B∆ contains a collection of input data and surfaces that are mapped back to K0

using calibrated associations. The associations help map fold-space to K-space. B∆

contains every known mapping from Φ0 to K0. B∆ is partially consstructed from

the results from the distortions of Φ0 after interaction with various training data.


B∆ is used when the surface is distorted by new data. Tolerances specified in B∆

can associate collections of newly measured values with knowledge representations.

Typically, to describe new data, items from a measure space are eventually

mapped to a non-measure space(M → N). There is no requirement that K-space

has no measures. If the destination form, K, already has measures, more of K can

be incorporated into Φ. B∆ holds the calibration that associates the measurable

surface perturbations with descriptions in K-space.

Though a region of an unfamiliarly deformed surface might seem to fall between

multiple familiar and similar deformations in B-space, it is not meaningful to inter-

polate interpolation in K-space. This is mostly due to the probable lack of measures

in K-space and possible non-linearity. Interpolation in B-space is also problematic,

as several distinct fold forms might map to the same K-form.

Augmenting knowledge forms

The high-level features provided as input to the framework, might not be able to

effectively (if at all) interact with K-space forms. Part of the creation of Φ0 is

the provision of interaction with D features. The data features were not created

with B-space in mind. The knowledge form is made effectively though indirectly

manipulatable, by bridging entities being augmented, malleable versions of the

knowledge.

Confidence: knowledge corruption

There is knowledge corruption in so far as the Φ0 form is different from K0. The

map-back should be tested for identity. Though an inverse map is notionally simple,

there is the issue of completely ‘unemphasized’, unperturbed surface, and whether

it can be mapped back. Though the folds have no corresponding tolerances (in the


absence of these folds and their corresponding K0 source, having descriptions from

B∆). The map-back of these folds need a similar if not identical map-back to the

ϕω folds.

The final description is Kfinal. Synthesis, to some extent, is the recombining of

the previously reduced space. Rather than trying to associate entire surface defor-

mations with complete situation descriptions, familiar fold shapes are recognized,

followed by synthesis of their associated sub-descriptions in K. The smaller K-

expressions will have varying confidence. Otherwise there is a single large expression

with a single confidence level.

Kfinal = Ψ(Kϕ), where ϕ is an individual fold (4.39)

As the notion of proximity is unclear in a K-space, proximity within measurable

spaces substitute, for example the surface (e.g. m nearest folds or folds of given

characteristics).

XD can be estimated from the proportion of familiar folds. The unfamiliar folds

can be associated using heuristics (particular to the expression language K), but do

not affect X greatly.

Mapping back to K-space configurations

KD is a configuration of the possible knowledge forms that could be indicators for

the recipient. There can be many such configurations of KD. There is not necessarily

one particular K-form for KD, as there can be a collection of K-form instances that

are acceptable interpretations of D.

Little further drift occurs if the map-back is 1:1, from B-space elements to K-

space elements, as the drift will already have occurred during iteration. Though


knowledge loss can still occur, if Φfinal is a better fit of KD than the mapped back

Kfinal.

Emergence also occurs after iteration when the recipient combines the presented

descriptions with other sources and their own experience. The purpose of the frame-

work is to present, to the recipient, an alternative more easily accessible collection

of indicators than the original data. The emerged “perturbed” surface from feature-

knowledge interaction is an approximation of the ideal description. Kfinal is the

configuration of mapped back K-space forms that the recipient has received to

interpret the situation. The collective fold information in B∆, as transformed back

to the knowledge space will help the recipient to gauge the significance of the original

data, than they could by examining the original data.

The remaindered surface ϕω contains the unfamiliar folds, which map back

similarly to the non-B∆, unemphasized folds that also map back to K-space. ϕω

needs to be treated as if it already occupies K-space. The resultant K-space

forms are likely to be previously unknown (unfamiliar). The K-space form of ϕω

can be presented to the recipient after discarding non-K-space constructs. If the

associations within B∆ were implemented as transforms these transforms could be

applied to ϕω. In any case the K-space-forms resulting, from remaindered surfaces,

would need to be distinguishable from those obtained with greater confidence.

Even if the final confidence values are low for a given configuration indicating

R (D), there might still be some benefit to presenting the mapped back K-form.

Arbitrary fixed tolerances could be used for the remaindered ϕω folds. Alternatively,

cleaning of the raw ϕω can be attempted, by removing the extra bridging form folding

structures. If B-space is not too far from K-space, the recipient will be able to view

somewhat noisy additional descriptions, appropriately accompanied by warnings of


low confidence. An option would be to present the raw and mapped back K-space

elements separately.

4.4 Summary

Surfaces, Φi, are metaphors for the internal representative forms of the bridging

entities Bi from Chapter 3. The situational referent is indicated by an entire surface.

Surfaces are perturbed when they interact with data. A surface is theoretically

comprised of a population of several folds ϕf , where a single fold is a partial

description or indicator of a sub-referent of the situation. A fold is considered

familiar if it falls within calibrated tolerances. The tolerances will have been specified

during prior calibration, and stored within B∆ (the experience store). Familiar folds

will be individually resolvable, and mappable back to the knowledge source K0.

There is notionally a fold ϕω which comprises all unrecognized folds. ϕomega can be

presented unmapped to the recipient, with suitable warnings corresponding to its

low confidence. If insufficient numbers of folds fall within tolerances, there can be

further interactions of surface and data until either sufficient folds are recognized

within tolerance, or the confidence fails to sufficiently improve.

The following chapter undertakes case studies in the context of two domains:

medical tomography and wildlife observation. In each domain, surfaces are used

to model the emergent results from interactions between general knowledge and

specific situational data. Fold sub-populations can be considered as representing

polyps in organs, unseen leopards, or more conceptually the natural phenomena of

predation. The case studies consider the initial assumptions and implications of the

transformation framework. They also reveal shortcomings, of the original theory,

and suggest improvements. Though the transformation occurs laterally between

different forms of reference, of similar referent complexity, progressively increasing


levels of situation complexity are used to examine aspects of the framework. Finally,

the chapter determines implications for and modifications to the framework.

Chapter 5

Situations

In this chapter, the framework is considered in the context of two disparate domains.

Assumptions, benefits and limitations of the framework are scrutinized by consider-

ing different levels of situation complexity. The situations are chosen to demonstrate

aspects of the framework. Desired aspects of the framework, such as the explicit

representation of formerly subliminal cues, are examined with respect to properties

needed for their realization.

From the perspective of the description, context is whatever influences the emer-

gence and subsequent interpretation, without being explicitly indicated by the de-

scription. Part of the context are the initial bridging entity forms, which the

interactions merely perturb. It does not matter if the context changes slightly,

if the interpretation is acceptable. However, the acceptability of the means of

estimating acceptability might be dependent on context not changing too much.

Domains are required in which context is not considered too variable for the purposes

of description. This is more reflected in the choice of domains where gleaned

information is more objective or pragmatic than subjective or artistic. Though

the domains investigated in this chapter vary in context between domains, within a

situation of each domain, there should be less variety.

138

CHAPTER 5. SITUATIONS 139

Aspects of the framework will be discussed subsequent to discussions with respect

to implications of the combinations of situational complexity levels and domains.

5.1 Discussing Situations

Each situation type is formally described, before being situated in the context of

one or more of the domains. The situations show how the framework addresses the

specific problem, given the peculiarities of each domain. The specific problem of

the thesis is the transformation of a reference of one form to a reference in another

form, while maintaining acceptable referent drift.

The main aspect of the framework is that it facilitates reference transformation

during the early stages of progressive emergence. It accesses a source of knowledge

(K0) and a source of experience (B∆). The model is that the experience enables

interpretation of the interaction of data and knowledge. It has been assumed that

prior calibration is required to make sense of the interaction, given the somewhat

arbitrary choice of prior feature extraction. Though the mechanisms of such cali-

bration are outside the scope of this thesis, the characteristics of such calibration

are inferred in this chapter. Interaction behavior is calibrated in the framework via

B∆.

One previously assumed characteristic is that there exists a corpus of data that

has accompanying authoritative descriptions. This is the basis of the tolerances in

B∆.

The referents are the situations in the domain that are referenced first by the

data, then progressively by the emergence results, and finally by the description in

the form familiar to the recipient.

The domain knowledge is that which is familiar and possible in the domain.

Unfamiliar situations will not have been described, in so far as calibration corpus


situations determine what is familiar.13 There is a possible exception in that ex-

pertly described situations that were rare in the corpus, might not have generated

tolerances for folds. A group of folds could be comprised of a single fold, if fixed

tolerances are being used.

In acceptable reference transformation, the description is an interpretation that

references a plausible situation. The situations investigate the consequences and

implications of acceptable transformation. The description domain is limited by

the representations in the nominated or adopted form. It is assumed that the

recipient of Kfinal makes personal associations between Kfinal representations, and

their experience.

There might be some drift due to contextual change across the data, however the

principle is that this will be considered part of the drift due to iterative modification

of Φi. Φi is the cumulative perturbation of Φ0 after i iterations. Modification

of the initial domain approximation of the situation description is by the seed

data interacting with the initial bridging surface. The initial bridging entity was

generated from the knowledge source. B∆ holds information for mapping between

K-space and B-space. The bridging surface is progressively perturbed. Continuing

legitimacy of fold tolerances and threshold confidence levels depends on non-abrupt

changes to context.

The representation of domain knowledge in K0 and B0 are references to what

is familiar or possible. The data is a reference to a particular situation (the data

referent). The desired outcome is another reference to that data referent in the form

of the domain knowledge.

13Familiar phenomena can be normal or abnormal. Familiarity relates experience, of previoussituations, to observation of current situations. Normality is a property of the domain. Similarly,unfamiliar phenomena can be normal or abnormal.


5.1.1 Situation Domains

Two domains are introduced to provide varying contexts for the application of

the framework. They are medical tomography and wildlife observation. Their

peculiarities help identify strengths and weaknesses of the framework. These do-

mains have, in common, the properties of continuity and data neighborhoods, where

unavoidable decomposition has already occurred. These domains have the benefits of

decomposition, without unnecessarily increasing the chances of losing useful indirect

interaction.

They differ in that medical tomography is more standardized, and is limited to

three spatial dimensions. The subject matter (the patients) help the observation by

lying still, in a particular orientation. There is a natural partitioning of the data

along each axis, with the possibility of recovery of some co-occurrence by processing

the other axes. Because of the nature of the natural reduction of the data, data

features used as input to the framework can maintain the relative positioning of the

raw data capture. For the most part there is a consistency in how organs are laid

out with respect to each other, and within themselves. There are three orthogonal

scan dimensions which can be calibrated separately, and yet able to be spatially

cross-referenced. Some data positions will suggest themselves as seed data sources

for starting progressive invocation of transformation.

The wildlife domain introduces a temporal dimension. However, the temporal

axis doesn’t have the same properties as the spatial axes of tomography. Though raw

data can be sampled at time points, data features can span time intervals. There is

no data that of itself is a clear candidate as being seed data for iteration.

For each domain, situations are considered that will test the ideas developed

in Chapters 3 and 4. The chapter concludes with suggested modifications to the


framework, based on implications gleaned from each situation, based on aspects of

the framework.

Each domain specification has a “Referents” and a “References” section. The

former states the kinds of possible situational referents; the latter the kinds of

situational references (both data and description). Both data and description are

references to referents. If there are multiple forms of data and multiple forms of

description, the distinction between data and description can be further blurred.

Though the delineation description form is more easily correlated with the data

than with the other description collection.

Medical Tomography

Doctors use raw multimedia, at very high resolution, to provide them with the

information they need [Wan08]. They interpret situations using their experience.

Tomography [ZZM+08][DA00] is used to image cross-sections of the human body.

Patients are oriented to match standard views, to avoid a potentially infinite collec-

tion of errors to relative position. The static environment of tomographic data allows

a reference frame to be used. This allows the system to apply reference coordinates.

The patient positioning error, is corrected by data cleaning.

The resultant images are used by radiologists to diagnose patients. A pathology

is a set of features or processes considered collectively; the typical manifestation

or behavior of a disease; an individual condition. The aim of the radiologist is to

determine the pathology of the patient’s situation. It is indicated by these features

(or processes), known as pathological features as they connected the pathology owing

to associations in the radiologists experience.

In some circumstances, medical specialists want their assistants’ assistance in

analyzing massive amounts of patient data. Often there is more data than both


the radiologists and their assistants can peruse. A data interpretation system,

implementing the framework, could help process some of the data. Though quicker

triaging of patients would be beneficial, the purpose of any filtering of descriptions

would be to reduce workload for humans. While termination of iterated emergence

would help reduce the time taken, the main benefit remains the avoidance of further

drift. Loss or lack of detail can impair interpretation of a represented or transmitted

situation. Medical doctors require high resolution in imaging to obtain sufficient

information to make diagnoses or control remote probes. The bigger picture includes

envisioning a situation where the human doctor is not available, that only an

automated system exists.

Tomography makes for a good study, as the data content progressively grows to

the limits of the tomography series. The discontinuity problems of shot boundaries

in video are not as marked here. Apart from the extremities of organs, there are

few sudden changes in context. Further, there is more information held about

the inner workings of a particular organ than the relationships with other organs.

So it might be better to consider the situation of a particular organ rather than

condition of the whole body. However from the perspective of the framework, it

should not matter at what scale you perform emergence. Information about the

significance of other organs and vessels, in close or far proximity, can be used.

The perspective of diagnosis, however, might require the same individual organ

detail even for situations which have sources of indication from multiple organs. It

would require the framework to deal with another magnitude of data quantity, if

not complexity. The wildlife domain shows the framework’s ability to scale with

increasing dimension and complexity.


The medical doctor can be drawn upon as an analog to the framework. Raw

scans are the feature data; the doctor’s mental model is the bridging entity; the

doctor’s report corresponds to K-space associated references.

Wildlife

Animals can be tracked and observed by sensor data systems which detect aural,

visual and other events[MMB+09] occurring in the studied environment. Motion

vectors of individual animals can be tracked if they have been collared or tagged

[JOW+02]. Large numbers of animals can be tracked using statistical techniques

[BHB+07]. As well as the three spatial dimensions available in the tomographic

domain, wildlife environments have an additional temporal dimension.

This domain contains some situations where the best indicators are not present.

For instance, the salient indicator of a predatory situation would be a sighting of

the predator itself (such as a leopard in a jungle) or sounds the predator utters. The

environmental data might well contain visible, audible and other sensor information

of other animals, but not of the leopard itself. The leopard itself (or an imagined

leopard much like the actual leopard in the jungle) would be the referent. A reference

suitable for the user would be a picture of a leopard (or other descriptive K-form)

perhaps in a diagram indicating a predator-prey relationship. The original sensory

data (or extracted feature data thereof) would be acceptable data referencing for

the needs of an expert.

The two situations refer to the same situation. One with a leopard sighting; one

without a leopard sighting. This is in the context of the expert naturalist whose

descriptions were used to calibrate the system. A different context might place a

premium on situations where the leopard has intentionally or otherwise allowed itself


to be sighted. The wildlife domain situations assume that the two refer to the same

situation.

Data forms (e.g. droppings, roars, paw prints) can be used in descriptions, along

with images of leopards. They can be inferred from animal herd motion vectors

and audible cries of potential prey. Herd vectors might be implicit in the data,

rather than actual motion vectors as data features in D. Separate collections of

detections can be associated if they share the same or overlapping time periods. Even

irrelevant data can be associated due to their large sub-populations. Data feature

input, whose corresponding extractors have extracted as much as possible, is more

likely to contain implicit or subliminal indicators, than data features from extractors

whose implementers have limited the extraction with a particular purposes in mind,

even if the particular purposes include interpretation of situations for the benefit of

recipients.

5.1.2 Situation Complexity

Nominating different levels of situation complexity allows progressive examination

of the properties and assumptions of the theory. The situations consider these

situation complexity levels in the context of domains to further determine properties

and requirements.

From the observer’s perspective, events which occur in the raw data environ-

ment can put the observer in mind of the other parts of the environment or prior

events. ‘In mind’ means the use of a recipient’s prior experiences to associate the

observed environment with other environments. The association can be with other

environmental instances of similar meaning, but not necessarily similar features.

The aim of reference transformation is to re-reference, the situations indicated

by the data, into the same form as the chosen knowledge source (the same form


as that used by an expert to describe a situation). The situations are for the

assessment of the reference transformation framework; sanity checks such as identity

maps, determining the requirements for acceptable reference transformation and

investigation of their consequences. More tests will be added if warranted by the

consequences.

The data features are approximate indicators of the events in the environment.

It has to be considered that individual features can straddle events in the situation.

We also have to consider that it doesn’t matter. In the first situation, the successful

processing of calibration situations is a desirable. At least in the case of the

calibration situational features, the description can be reached despite straddled

features.

Further implications, for the framework, are inferred from the needs or char-

acteristics of the domains. When the peculiarities of the situation domains show

behavior different from that predictions of the theory, two statements are made.

The first comprises the limitations of the theory in terms of what ideally should

have been possible. The second is comprised of advisable adjustments to the theory.

Wherever possible, determinations are made of what is either achievable, achievable

within limitations, or not achievable.

5.2 Domains

5.2.1 Tomography

Referents

There are two levels of referent: the general pathology of a kind of organ, and the

specific pathology of the particular organ in the current data. Specific pathologies


are pathologies that have not, as yet, been accepted as ground truths (not even

ground truths nor implications for particular contexts)

In the data, there exists spatial variation due to differently sized people and

differently sized and spaced organs. Human radiologists can make allowances for

this. The human radiologist should be able to spot that an organ is distended

somehow, though this can be a subliminal effect that combines with their experience.

If the distension is explicitly noticed by the expert, the training data should have

instances where the descriptions mention this. If it is not explicitly noticed, this is

possibly something which to which the system can draw the recipient’s attention.

Eventual interpretation of organ data needs to be tolerant of variations of the same

kind of organ, while also differentiating on specific abnormalities. Tolerance needs

to account for natural variance of the same organ (referent), as well as events of

similar significance.

There can exist several referents: spinal column, organs, notions of normality or

abnormality. These several referents can compose either a single referent pathology,

or multiple pathologies. As the system is not informed of a referent cardinality, the

interactions can include an interference effect from indicators of different situations.

References

The tomographic sections are the initial references; the description fragments are

the final references. Both purport to indicate the specific pathology of the organ.

The nature of the data capture, leads to a partitioning into three series of

sections; one for each coordinate axis (aligned along the body, face-on and side-

on). Each set is a different view of the body; each can participate in emergence

separately. If the three separately obtained final descriptions agree, that is the best

that is possible. If two of the three concur, it is still acceptable but with lower


confidence. If they all differ greatly, then it is difficult to apportion much confidence

to any.

Peculiarity: Tri-axial data series

The description(s) of the domain pathology is used in calibration three separate

times, one for each axis. The feature data interacts with the same B0. B0 is a

copy of Ka. Ka is the knowledge source equivalent of a medical atlas; it is not a

typical medical atlas. Ka is comprised of pathology descriptions of the organs and

possible ailments, which includes spatial measurements. Sectional data is stored

with the sample organ data. There is no technical reason (storage capacity aside)

why all patient data and diagnoses couldn’t be stored. This does not require a

human expert to sit down with the system. A teaching system for students can also

be used to teach the system. Actions performed with surgical instrumentation could

be logged, so the descriptions need not be natural language reports. The diagnoses

are just useful interpretations in forms other than that of the original reference

data (the tomographic scans). The diagnosis fragments generated are appropriate

or acceptable alternate-form references.

Abnormalities and tissue densities are the main things looked for.

The training of the system interacts B0 with the data from each organ instance.

The results of interaction are correlated with the diagnosis of the organ(s). As the

training has been performed on data from many patients, the calibrated knowledge

source holds tolerances about the data (and possibly also tolerances about the

diagnoses if different experts differ on diagnosis).

Diagnosis can be more than just labeling or classification. External filtering by

the recipient14 would take as input - the description fragments of Kfinal; not simple

classifications. The limit as sampling approaches completion, of data displaying

14The medical practitioner or an automated process in the calling system.


a known pathology (e.g. sections of a known organ), should result in at least

classification of a known pathology in terms of the KB. More usefully it will provide

diagnosis fragments in expressions of K-space.

A confidence threshold might not be a simple fitness value. For acceptance

of a calibration instance, all other patients should still have diagnosis correlation

even after changes based on “acceptable” description, of the current and subsequent

patient(s). An initial threshold to deal with an initial corpus of calibration data

has to be somehow contrived for acceptance of the folds with tolerances. The

superpositions of folds resulting from multiple patients must realize tolerances that

act in concordance with thresholds. The advantage of the medical domain is that

many patients’ data will exist so that the normal and the familiar, in the larger

population, can be represented/expressed from the corpus. The sparse if not disjoint

nature of interactable data features and knowledge source elements, will result in

a threshold determination that can be computationally justified. The complexity

of a medical condition could necessitate the separate iteration and termination

thresholding for many small diagnosis elements.

The body and constituent organs provide a well known environment. The range

and combination of normality is more constrained. It is analogous to all wildlife

detection regions having an expected “normal” configuration of landscape and flora.

There exists a much higher confidence that calibration data is representative of the

population. This relatively high domain confidence is not built-in to the framework.

Dx is the collection of visual features for the x-axis. It is broken down into

tomography sections, di, which are further broken down into features. The seed

section is d0. The subscript of di is the distance within the data from the seed

section.


��

��

��

��

��

��

��

��

OUT OF SCOPE

IN SCOPE

IN SCOPEOUT OF SCOPE

Bxn

. . .

Bx1Bx

2

Mini-diagnoses and -descriptions

Selection of section(s) dxn

from tomographic x-axial series

Bx0

. . . . . .

d−xn−1 dx

n−1. . . d−x2

d−x1 dx

1

dx2

dx0

Selection for Transmission and Presentation

Synoptic Diagnosis

Bx3

Map to Organ Descriptive Form

∗ Fragments of Organ Description

Composition⋃

korgan.desc

Final Diagnosis for Transmission

Reasoning

. . .

Figure 5.1: Emergence of Organ Description

Dx =

d0 : {f0a , f0

d , f0e , . . .}

d1

d+x1

d−x1

{f1k}

...

dj

d+xj

d−xj

{f jk}

...


{f jk} contains features from both of the sections d+x

j and d−xj , which are j sections

away from the seed section.

The data comprises three image series of an organ. There are three axes x, y

and z. The respective image series are Dx, Dy and Dz. The triple series for each

patient allows tri-axial verification. The set of three series of tomographic data (one

per axis) should approach the same description or interpretation.

Daxis ≡ {daxisj } (5.1)

The data relating to a single instance of an organ comprises {Dx,Dy,Dz}. It is

referred to as Dxyz. Multiple instances of organ data (involving either a single organ

or a region of multiple organs depending on need), including three series of axial

data for each organ, are used. However calibration is stored for each axial Daxis.

3-axis agreement deals with consistency. However, this could be gained by

mapping everything to NULL or homogenous noise or a lack of deformation from

the representation of general knowledge.

For a single activity of emergence, a single axial series of a single instance of an

organ, is involved in interaction.

Dx = d−xm1 . . . d−x

2 d−x1 dx

0d+x1 d+x

2 . . . d+xm2 (5.2)

The negation on some subscripts is just to indicate the direction from the first

stage data dx0 . The data introduced in invocation j dx

j ≡ {d−xj ,d+x

j }. m1 and m2

can be different as the initial data dx0 is not necessarily in the center of the data

series.

The reference that eventually emerges from the interpretation of Dx is Kxfinal.

Dy and Dz are independent runs that produce Kyfinal and Kz

final respectively.


Ideally, after verification:

Kxfinal = Ky

final = Kzfinal (5.3)

B∆ is partially comprised of processed knowledge from a corpus of tri-axial

series of tomographic scans. There are tolerances for each axial series. Metadata

will specify which axial data from B∆ for calibration of the interaction of B0 with

the current situational data. B0 can be the same for all axes, if it and Baxisfinal

are description of the domain and pathology rather than annotation of the raw

data. However, any calibrations would be different for each axis. The different axial

emergences will evolve hypotheses at different rates. Their termination depends on

the distribution of events in the data, which can be different along each axis.

Different calibration instances of the same phenomena (i.e. scans, of the same

organs exhibiting similar pathology, from different people) help establish tolerances

of known pathology. Some medical descriptions will assume a basic knowledge of

the domain, and will concentrate on abnormalities. This addresses null results and

noise, but doesn’t deal with the general knowledge (unless of course the patient data

correlates with knowledge of known phenomena).

Tolerances in B∆ specify the acceptable bounds of familiar behavior, for either

normal or abnormal organ instances. The spatial tolerances of an organ should allow

for relative sizes of organ instances; for example adult organs relative to juvenile

organs.

The confidence (X ) and tolerance (τ) will be affected by the number and the

variety of organ instances in the calibration corpus. The predictive confidence Xfold

in a particular fold or description will be based on the variety of examples of that

condition.


It’s possible to reach a point in the data series which triggers X ≥ Xthreshold.

However, this doesn’t allow us to say that that point is important. It could be

that a collection of points is important or that a region already passed was salient,

and this merely removed some uncertainty (e.g that nothing was important at that

point).

If another arbitrary start point were chosen, and the prior point triggered the

confidence threshold, it wouldn’t prove that that prior point was important. The

locality of interest or abnormality would be the entirety of the data seen thus far.

Though the trigger point could be presented to the the user for consideration.

However, if interaction with a data extent generates a polyp-like description (all

the characteristics of a polyp) in an area where polyps are not expected, this can be

flagged for the recipient of the K-space description fragments. The recipient could

further infer or verify a strangely located polyp. If the spatial knowledge for polyp

location does not affect the resulting folds, a false negative (of an abnormal and

familiar event) occurs in that insufficient newness is discovered.

5.2.2 Wildlife

Referents

The referents of wildlife can be a relation such as predator-prey, a mood such as

serenity, or the presence of creatures. Alarm is more likely to be explicitly described

in the known situations used for calibration. A leopard in a jungle could participate

in any or all of those situations. Salience depends partially on the expert descriptions

of prior system experience. With the added temporal dimension comes the notion

of events within a passage of time.

The description is not a classification problem for leopards. That could be a

(notionally) simple supervised training exercise for a sufficiently complex neural


network. The aim is to describe observed facts and associated implications. As all

data is interacted, some parts of the description (while accessible) might be neither

desired by nor useful to the recipient. Referents could be the conditions of a given

location, somewhat like what some content management systems call metadata.

Though what is considered data and metadata would be dependent on the external

purpose of the calling system or recipient. Simpler sub-referents would be expanses

of terrain and jungle.

The original data referent is the notion of a leopard in an expert’s mind. The

ideal final knowledge referent would be the notion of a leopard in the recipient’s

mind.

References

There exists a collection of sensor data gathered over an observation period. It

is seen as a neighborhood in terms of progressing time intervals. There exists an

observed territory in which sensor data (including visual and audible events) is being

detected and recorded. The spatial dimensions of the territory and the temporal

dimension combine to provide location metadata, which can also interact with K0.

Data references include calls and motions of birds and animals. Possibly in-

volving co-occurrence of events from different media. For use with the framework,

there is no importance in an event being visual, audible or other. However, for the

purposes of these situations, they are convenient subsets of the event population.

Other data items can include geographic features and lighting, if the sensors can

provide them. These are context only insofar as their co-occurrence affects the

interpretation, due to all data being potentially capable of modifying a bridging

entity.


A single data feature can be an entire deer, where the feature is an output from

a vision system.

Significance of event rarity, regularity and sequence are held in K0 and B∆.

The time interval between subsequent sensor readings, provides a granularity

analogous to the section separation in tomography.

To an observer, flying in a plane, terrain would seem to be observed hues and

textures. An observer at ground level would see individual leaves of individual

trees of jungle. There are different gradations of detail. Having separate experience

stores, (B∆)observer, and separate interactions would limit interference and misin-

terpretation. However, future changes (possibly improvements) to data capture,

could perturb the bridging entity beyond tolerances for some descriptive elements

(possibly degrading the amount of description and/or the confidence).

An inferred leopard is described not based on its immediate environment, but

from the detected outer environment. It is only if the calibration leopard was directly

detected, that the detection events can be used in the description. To keep the form

of description consistent, another sample leopard could be substituted as part of the

description. The knowledge can be in different forms. New descriptive forms can

be part of the recalibration. This does not affect the perturbation of the bridging

entity.

1. Temporal data can span the natural partitioning

The data features provided might span the natural partitioning. For example, a

axis for iteration could be time. The natural partitioning would group data from

different points in time. The same deer can populate multiple data time points.

Motion vectors for the deer could span multiple time-wise data partitions.


Linear distance measures are sufficient for both data features and fold measure-

ment - for both dwildlife and ϕwildlife.

There are many kinds of animal behavior, so much so that some behaviors will

not have been captured within B∆. There will be some significant cues that are

missed, not understood or misinterpreted.

Future calibration research will need to investigate the association of particular

folds with portions of an expert description. The recipient of the framework must

assume that significant missed data still have interactions whose results should

have been associated during calibration. This can contribute to incomplete fold

generation or insufficient fold emphasis.

It is hypothesized that prior calibration has passed the requirements of recreating

acceptable descriptions from calibration data instances. If it is further hypothesized

that subliminal data might not have been involved in interaction, variations in the

missed subliminal data will not have affected the folds created. Significance of the

occurrence or co-occurrence of the subliminal data will be missed in the folds of the

perturbed surface. Further, there might be multiple expert descriptions provided

for ostensibly the same data.

The data features interacted cannot be restricted to phenomena ‘known’ in the

calling system, or even ‘known’ by the expert. There would be no significant

difference to rule-based systems. The expert is not allowed to nominate trigger

events (e.g. deer scatter).

There are possible temporal parallels with ice-scape area delineation. The leop-

ard event can occur temporally before the symptom events. The (past) undetected

leopard, the (present) inferred leopard, and the (present) detected leopard are

different events.


Causality of events is estimated from causality information in K0 and data event

time sequences. Persistence of an event in the temporal dimension (if part of the

slice dimensions) could cause either a proportionality significance description or

merely an “exists” description fragment. This would be true as well for persistence

(repetition) in spatial dimensions (when an animal or plant “stands still”).

2. Multiple media

The framework will allow indirect interaction of the bird calls and deer scatter. It

does this by interacting both with the bridging entity that was initialized by the

jungle knowledge source. However the interactions should occur in parallel. Either

interaction sequence is unlikely to yield the same results as the parallel interaction,

and possibly not even to each other.

(bird calls Υ (deer scatter Υ jungle knowledge))

6= (deer scatter Υ (bird calls Υ jungle knowledge))

(5.4)

This is a processing practicality for non-parallel systems. Furthermore there will

be interpretation drift due to different start times. They will have different temporal

extents; imperfect overlap. Though the overlap imperfection is the same for both

calibration and situation runs. Could get the same start (seed) time from trigger

events, such as motion sensing in wildlife photography (e.g. catching snow leopards

on film; lightning sensing photography).

The rapid movement of a herd or many individuals could be more significant

than a single animal’s behavior.

Each medium in the wildlife domain can be an analog of the visual events in the

tomographic domain. The three spatial dimensions of tomography are now joined

by a temporal dimension. Sensor network data (of wilderness observation) can also

be taken in slices. These can either be defined by a subset of dimensional axes, or


be the n-1 dimensional data perpendicular to a given axis. Interaction with K0 will

require calibration of B∆ for each set of dimensions in a slice.

Though sound events and vision events co-exist in the same dimensional data

space, there does not have to be a direct mapping between sound and vision events,

however there could be expert calibration for each medium’s events interacting with

K0. Both visual and auditory data can be mapped to K0. If multiple media co-occur

in the data being described, there need not be a way of linking events.

If the importance of the co-occurrence has been captured by the calibration

of B∆, an isolated event of a group of events should not provide high confidence

for emergence of description fragments. This is unless the isolated event has been

separately calibrated either in isolation or with another group of events.

5.3 Levels of Situation Familiarity and Complex-

ity

Before considering the domains in the context of very complex situations, the do-

mains are considered with less complex situations, where simplifying assumptions

have been made. Levels of situation complexity are examined in increasing level

of complexity, with each subsequent level subsuming the former. The number of

assumptions reduce with increasing situation complexity level. A familiar situation

assumes that the entirety of the situational data has been previously experienced

by the framework. An unfamiliar situations could still have familiar elements.

Emergent folds will have been individually encountered earlier, but not collectively.

Partial familiarity occurs when only some emergent folds fall within tolerance. For

Sparse data, data references are spread thinly or unevenly across a large extent of


the data. Finally, a complex situation allow for indicators to lie across extents of

data, overlap and possibly indicate multiple concurrent phenomena.

Each complexity level is introduced formally, and is then discussed in the context

of one or both of the domains under consideration. Temporal concerns are dealt with

if appropriate in each situation. The situations assume that surfaces are being used

as the implementation of the framework. Concepts, such as the bridging entity and

emerged results from interaction, are couched in terms of surface notation.

The framework considers referent maintenance

R (D) = R (Kfinal) (5.5)

and the termination of invocation of emergence via either threshold or equilibrium.

X (R (ρn) = R (ρ0)) ≥ threshold (5.6)

ρn ≈ Ψ(ρn) (5.7)

Information loss in the context of the framework, is the loss of cues, explicit to

the recipient even in the situational data reference. Data reference cues that aren’t

appreciated by the recipient, that are also not made explicit in the description

form aren’t losses15. If references in the data are not mappable to K-space by the

information in B∆, there should be some indication of the incompleteness of the

description.

There is a special significance, if the resulting reference in the destination form

was not easily appreciated by the recipient’s observation of the source form. The

system is capable of capturing implicit or subliminal cues. This is the aspect that

considers emergence as converting the implicit to the explicit.

15Except in the sense that there was a lost opportunity.


5.3.1 Familiar Situations

Ideally the situational data from a situation used in prior calibration would interact

so as to reproduce the expert description corresponding to that calibration data.

This would be part of the process of training or bootstrapping the initial experience

store B∆.

Interpretation of the situational data collection Dj, is denoted I (Dj). In most

cases this would be identical to the Kfinal for Dj .

If it is supposed that the situational data collection Dj was used in the produc-

tion, say by calibration, of the auxiliary entity B∆, it will be denoted Dj∆ as well

as Dj.

Given that

Dj = Dj∆ (5.8)

it is desired that

I (Dj) = I (Dj∆) (5.9)

Unfortunately, the interpretation of Dj∆ was defined by the corpus of expert

associated data collections and K-space descriptions. I (Dj∆) is not necessarily the

same as the Kfinal for Dj∆.

The Kfinal for Dj∆ would have been generated for the purposes of setting

tolerances of folds, and establishing mappings from B-space to K-space.

Though the existence of interaction between knowledge and data elements is

assumed, there is no control over the results of such interaction. The control available

to the calling system’s oversight of calibration would have been in the acceptance

or rejection of the “feature extractor and data corpus pair” for the purposes of


calibration. It is the notion of interpretation acceptability that is crucial to the

acceptability of feature extractor and data corpus.

This situation type (known as “Familiar Situations”) assumes that tolerances

within B∆ were generated by some means of calibration. While the mechanism of

the posited calibration is not within the scope of this thesis, the characteristics of

action and results of calibration are inferred or implied by what are deemed to be

additional requirements of the calibration.

For acceptance during calibration, the identity process would have needed to be

deemed to be acceptably close, though not necessarily exact in recreation. However,

the determination of acceptability is crucial to the operation of an implementation

of the framework. It is assumed that tolerances will have been enhanced by multiple

mappings to the same description.

It is desirable that the Kfinal of a Dj∆ be either acceptably similar to the expert

description associated (with Dj∆) in the original data corpus, or acceptable as

an interpretation according to the original expert. As one of the benefits of the

framework is not requiring the presence of an expert, at least for Dj∆, acceptable

similarity will have to have been determined.

The results of domain knowledge and situational data interaction need to be non-

trivial. They should be of sufficient complexity so as to not necessarily lose informa-

tion, even if the information is not apparent. The function of the B∆ mappings are

to attribute meaning in terms of K-space forms. If calibration was used, its purpose

would be to establish the folds and tolerances of B∆. Post-calibration, there exists a

system based on sufficiently varied data instances, such that an “informed” source of

tolerances have been generated. If threshold confidences are available these should

be comparable16 to the generated confidence level of the latest bridging surface.

16In the sense of being able to be compared.


Dj∆ is not necessarily stored in its original form in B∆ past the stage of calibra-

tion. There would be no reason to suspect that Dj∆ would need more interpretation

than that provided by expert description. The invocation of the framework by a call-

ing system would be for situational data collections for which no expert description

is available.

Since overall emergence occurs as a result of a succession of individual emergences

during neighborhood traversal, an interaction is more accurately portrayed as

Dji Υ B0 =⇒ Φji (5.10)

where the seeding of the overall emergence begins at data item i , (di). Emergence

can be seeded at different points in a data set. The emergence from other seed points

can be used to verify the result of a given seed point. Usually a seed point is arbitrary,

though properties of the domain might suggest one or more sensible seeds.

A stronger approach to calibration would have been an insistence on a wider

range of seeds. The same interpretation as the experts can not be guaranteed

unless the same seed section as calibration is used. The multiple folds generated

will collectively map to an acceptable complete description if only because the

composite mapped description has been authored by the expert before calibration.

In section 5.3.2 and beyond, acceptability will depend on a confidence measure.

There is no general guarantee that each situational data collection will perturb

the bridging entity such that the appropriate fold tolerance is met. It is expected

that most emerged folds would match, as the tolerances were derived from the

resultant folds of calibration interaction. However, if the tolerances are not simply

spaces that encapsulate all variations of the folds from calibration, it is possible (in

some domains) that not all emerged folds match.


It is also possible that a fold might satisfy the tolerances of multiple B∆ folds. A

region of folds in the emerged surface would trigger associative mapping from differ-

ent configurations of folds-with-tolerance in B∆. This is dealt with in section 5.3.5,

which covers multiple phenomena in situations.

The transformation system produces a description as a natural result of the

emergence, and consequent mapping based on information in B∆. For an arbitrary

domain there is no expectation of reproducibility of interpretation for data collec-

tions that were used during calibration.

Tomography

B∆ is assumed to have been calibrated on the data and results for n separate persons,

with 3 separate axial tomographic series for each person. Each Dj∆ might have been

a patient whose scans were used in a training system for students. Multiple axes can

be used for verification; there are existing techniques for combining evidence and

confidences. B∆ would have been calibrated on multiple persons and an archive of

the interactions of each person’s corresponding data with K0.

For simplicity, it is assumed that each emergence for each calibration data

instance was seeded from the central section. The main issue here is what confidence

the system places on the interpretation I (Dj).

Semi-arbitrary seed points can be the center, top or bottom of the organ (or of

body region examined). Starting from the center of the axial data, equation 5.2

becomes

Dx = d−xm . . . d−x

2 d−x1 dx

0d+x1 d+x

2 . . . d+xm (5.11)

The central section of a data series is not necessarily the best, though it will

mean that the extents m1 and m2 will be equivalent.


The triple sets of axial data from each Dj∆ will provided information to the toler-

ances and mappings contained within B∆. Three separate approximate descriptions

of the same training situation will have been available. There would have existed

differences between the three intermediate (surface) representations (Bx0, By

0, Bz0),

though there would have been no difference in K-space representation (as all have

the same associated expert description). The axis chosen is not interchangeable. The

axial positions of the sections while not interchangeable can be useful. The spatial

intersection of the sections (and sub-series of sections) can help locate volumes of

interest.

A description of sufficient confidence can be composed from description fragments

associated with folds in B∆. This is because each fold generated from Dx will be

within tolerance for a composite fold specified in B∆.

If a single instance of an emerged fold, only takes part in a single composite fold

then the tri-axial verification will be trivial. This is because the all the composite

descriptions will be acceptable in light of the original expert description for Dj∆.

All tests

(X αD≥ X α

threshold, α ∈ x, y, z) (5.12)

will pass. And the test

(I (Dα) = I(

Dβ)

) (5.13)

where α, β are two of x,y,z) will pass for all pairs α, β.

If an emerged fold can satisfy tolerance for multiple composite folds, then the

test (I (Dα) = I(

Dβ)

), where α, β are two of x or y or z, could fail for all axial

pairs α, β, as the same K-space forms (Kfinal) will not be guaranteed for all axes.


Wildlife

An interaction was denoted in Equation 5.10.

Dji Υ B0 =⇒ Φji (5.14)

Dj can be the time bounded section of data stream of wildlife sensor capture

that has a particular description associated with it. This will have an error corre-

sponding to the discrepancy between the expert’s nominated temporal section and

the temporal extent from which they ascertained their description.

The choice of seed point, di corresponding to Dji, will need to obey some

previous-to-calibration determined heuristic, such as seeding at d0 or dSeed, where

Seed = function(tstart, tend) (5.15)

where tstart and tend are the times at the start and end of the expert described

data stream section.17 As an example, placing the seed at the midpoint:

function(tstart, tend) =tstart + tend

2(5.16)

is straight forward for the calibration data, but problematic for normal situa-

tional data as there are no obvious tstart and tend.

The data for a point in time dtimepoint makes little sense, for animal behavior

as movement and sound information are lost, unless a time point is associated

with the data that spans time between time points. A particular di for wildlife

represents a time interval, whereas a particular di for tomography represents a

17Analogously in current text description schemes, tstart and tend would be the ends of anMPEG-4 data stream.


position. Successful calibration is assumed for the use of the framework post-

calibration.

5.3.2 Unfamiliar Situations; Familiar Folds

This type of situation is characterized by the entire description of a situation being

representable by familiar K-space forms. The framework is primarily used for

lateral reference transformation, by interacting familiar knowledge with unfamiliar

data. Each emerged fold satisfies one or more existing B∆ folds, within tolerance.

An assumption is that at the point of termination of iteration (if any), emerged

situational folds collectively cover (span) the emerged situational surface. There

are no (or negligible) emerged folds that fall into the indeterminate regions of B-

space let alone the distinctly unknown regions. The confidence estimation technique

would assign a comparable confidence level to the surface interpretation as any of

the individual folds.

If a mapping exists from each fold to a K-space form, then a composite descrip-

tion can be presented to the recipient. However, for the tolerances and mapping to

be acceptable, there are implications for means by which B∆ was produced.

If calibration from a training corpus was used, then the issue of transfer of

authority to from the entire surface to each B∆ fold would have needed to have

been resolved. It is assumed that there were sufficient data points in the corpus,

for calibration to enable folds and tolerances to be matchable with future emerged

folds. For the data collection instances satisfying this situation, the collection of


familiar folds is complete.

D1∆ Υ B0 =⇒ Φ1∆

...

Dj∆ Υ B0 =⇒ Φj∆

...

Dn∆ Υ B0 =⇒ Φn∆

Calibration:

setting tolerances

and mappings

→ B∆ (5.17)

Φj is the surface that resulted from the interaction of the jth calibration data

collection (Dj∆) and the initial bridging entity (B0).

Φj∆ ≡ {. . . ϕjf . . .}∆ (5.18)

In Equation 5.18, each surface Φj∆ is the collective perturbation of B0 after

interaction with Dj∆. It notionally corresponds to a collection of several folds where

each fold ϕjf maps to the K-space. The folds are illusory until the “tolerances and

mappings” of Equation 5.17 have been inferred from the correlation of Φ1∆ ... Φj∆ ...

Φn∆ with KD1∆... KDj∆

... KDn∆. The emerged calibration surface does not have to

survive in the final B∆ that is used for situational interpretation. Where possible the

description fragments exist at a granularity that allows for distinguishing situations

that would be understood as different situations by the recipient. This is dependent

on the granularity and variance of detail from the original expert descriptions that

are resolvable in K-space.

As folds can overlap, the collection of resolved folds is not a partitioning of the

surface. There does not have to be the same number of resolved folds in each surface.

Furthermore the intersection of fold collections can range from empty to equivalent

to one of the fold collections.


Dj Υ B0 =⇒ Φj : {. . . ϕjf . . .} (5.19)

Its constituent folds are made up from folds possibly from anywhere across the

entirety of B∆. The final description, in forms that the user can understand, is

composed from description fragments (K-space sub-forms).

K − space − forms

Bridging Entities

Fragments of expert description

Folds within tolerances

(a) (b)

Figure 5.2: Composition of final description

B∆ contains the calibrated association information for mapping from the resolved

folds in Φi. As each emerged fold is accompanied by a confidence, the final descrip-

tion can have an overall composite confidence. The composition of overall confidence

is perhaps not generalized for all combinations of co-occurring folds, especially when

there exist non-linear relationships.

It is important to note that the folds do not need to correspond one-to-one to

data features. Markedly different data features interactions can satisfy the same B∆

fold.

The same sensor data can have two expert descriptions, where both are con-

currently acceptable. Within the data there can be two separate situations. Of

the description fragments generated, there would be sufficient fragments to describe

both situations.


Tomography

Folds corresponding to polyps occur, but in regions not previously known to contain

polyps. Similarly, other familiar symptoms of organs appear, but in location,

distribution or concentration previously unknown.

Equation 5.19 becomes

. . . Υ (dj Υ (. . . Υ (d0 Υ B0))) =⇒ Φj : {. . . ϕjf . . .} (5.20)

dj is the current tomographic section from the data instance being interpreted

via emergence. d0 is the seed point for the iterative emergence that is summarized

by Dj Υ B0.

Positions, in an organ or a body (within a given error owing to alignment or

variance in organ and body instance sizes), can be inferred from scan numbers

and the known scan separation distances for the scan axis. Three dimensional

positions can be inferred from the three separate scan axes. This inferred positional

information could either perturb each interaction or be representable within the

knowledge representation.

Wildlife

The context of the naturalist’s interest can be limited to a particular area in the

wilderness. This is analogous to a description stream element being limited to a

specified extent of a media stream element. However there are no rigorous correla-

tions or candidate correlation sequences, apart from the the passage of the observer

(for example, a plane flying above terrain). Similarly, in tomography, the radiologist

“flies” along an axis of scanning. This is a form of spatial modeling with regard to

“location in the region” of the predator.


The capture of non-inferred leopard description references can be correlated with

other data within a temporally delimited neighborhood. The path can naively be

considered to be through time, if temporally spanning data features are ignored. the

naturalist traverses a time-line of sensor capture.

Inference of the proximity of the unseen leopard depends on the nature of the

unseen data. The source populations for fold groups are likely to be highly varied.

Data features such as multiple moving deer. The number of deer in the herd, and

the relative bearings of the warning signs can affect the sensory data.

Multiple events can include multiple instances of the same type of event or

different types of events co-occurring. A leopard can be hunting deer at the same

time as a crocodile is hunting wading creatures such as deer or water birds. A data

collection, D, contains bird calls from one calibration instance and deer scattering

from another instance. These varying data events can correspond to the same

description references indicating fear or predation. Other parts of the description

will fill in details of what might be taking part in the predation or expressing fear.

Both sunset and predation phenomena can occur in the same region of observa-

tion. Both can be described, and to some extent need to be distinguished by the

recipient who receives the description. The calling system can perhaps filter the

K-space descriptions after invocation of lateral reference transformation.

There can be co-occurrence of types of indicators of leopards not normally

coincident. Or previously coincident indicators that are coincident with different

relative timings.

Not all simple interactions (Bn and a single feature) will be equally significant,

or even significant in proportion to the amount of interaction. This also applies to


simple group interactions (Bn and a group/pattern of features), aggregate interac-

tions and complex interactions (Bn and different sets of features which seem not to

have similarities or shared patterns).

A single startled deer, multiple startled deer or startled animals other than deer

should result in descriptions of danger or predation. Their respective surfaces will

need to be within tolerance.

To distinguish deer scattering as opposed to deer walking, the temporal axis

can be used. It is the temporal axis along which different data instances are

distinguished. Motion vectors of deer scattering are illusory. It is something that

would either need to be calculated, or already present in the data features. The

isolation of the deer as individuals are already be present in the input data.

While there can be absolute position within a given sensor region, there is no

absolute position along the temporal axis. Events can occur at any point along the

temporal axis.

The temporal nature of wildlife sensing, can be addressed by continual invocation

of emergence from successive data captures with progression in time. This mirrors

the parallel content-description streams in video standards (for example MPEG-4).

Each resolvable time point can be used to seed emergence with iteration in either

spatial and/or temporal axes. Vectors, with respect to time, are not explicit in a

reduced data feature collection. Though movement of animals might even be obvious

to a recipient who doesn’t appreciate their significance.

Tomographic data provides repeated situations where many known normalities,

can be expected as existing in calibration data unless the expert descriptions state

otherwise. The familiar normalities outnumber the familiar abnormalities. Toler-

ance can be set more confidently. Wildlife data needs a greater correlated binding

between description stream and event stream.


Implications

This situation type assumes that it is possible to describe new situations, simply by

using satisfied folds and mappings, provided in B∆. So there must be some re-usable

elements either from prior K-space descriptions, or from the mappings themselves.

Some elements of a new situation will be familiar.

Some descriptions are more appropriately coupled to specific spatial regions

within existing data. It could be that the descriptions are only valid for some

regions. However, in that case, the delimiting of validity has to self-partition

naturally, and/or communicate to the calling system sizes of spatial regions for

which emergence can be valid. There are some descriptions that would be for an

overall feel of the environment.

Thus far there has been no consideration whether description forms are related to

input data forms. Sometimes cumulative or accreted data references are the domain

appropriate improvement of reference.

Theoretically there is no problem with this, as the framework could (appear

to) simply transform aspects of the data that are obscure to descriptions that are

complementary to the rest of the data. It is the reconstruction or preservation of

the remaining raw data that is problematic. Ideally, the persistent data appears

alongside modification to the progressive description, though as an identity map

of part of the data reference. Calibration descriptions would have preserved or

embedded the raw familiar data. There is no distinction between annotation and

preserved elements. There is a danger that calibration data will erroneously take

the place of situational data or description. If there is no control of the interaction

result, there is no guarantee of eventual creation of a description that seems to

embed the data. And no guarantee that a description will embed situational data

rather than calibration data.


Described delineated regions are akin to labeled data if the complexity of the

data is much greater than the complexity of the augmentation, even though the

resultant combined reference form is technically at least as complex as the data

reference. The perceived description is just a classification if that is all that is

required. The framework is advantageous more for domains where the description

reference is significantly different in form to the data reference.

The delineation, of data regions for which description is valid, is analogous to

parallel data and description streams for media that has one clear traversal path.

Descriptions are effectively superimposed onto the areas of validity. The confidence

of emerged description is associated with the interpretation of a delineated region

within the input feature data. Delineations can overlap with respect to the data.

For tomography, there is an assumption of validity for an entire section. For

wildlife it is less certain that the entire region of detection is to be described. It

could be that whatever the entire enclosing region from which data is obtained is

the region that is being described.

There does not need to be co-occurrence of data within a section or between

sections, so long as the folds are within tolerance. There will have been loss

of information due to reduction of non-linearly combining data, and also data in

the scan separation distance between sections. Any surviving non-linearity will be

contained within the interaction mechanism.

5.3.3 Partial Familiarity

The complexity level increases further, with the consideration of unfamiliar situa-

tional sub-referents, or even previously encountered concepts indicated by unfamiliar

folds. “Partial familiarity” is a special case of multiple events indicating a single

situation. The collection of calibrated folds in B∆ can be incomplete with respect to


spanning all possible surface perturbations. Φsitu is the resultant surface emerging

from Dsitu Υ K0, where Dsitu is the situation being interpreted. Not all folds of Φsitu

will fall within the tolerances of the B∆ folds. At least one of the folds, emergent

from the interaction of situational data and knowledge source, cannot be found in

any of the B∆ folds.

From equation (3.12),

B∆.(mappings) = Γa

(ba) (5.21)

bj is a binding association between ϕj and a K-space form.18 Γj

simply indicates

the collection of binding associations, without implication of aggregation.

Φfinal = Γiϕi + Γ

jϕj (5.22)

where ∀i ϕi satisfies tolerance; ∀j ϕj does not satisfy tolerance

Part of the situation can be described by the combination of known folds.

The final composed description is created from existing authoritative description

fragments and raw emerged situational description folds.

Kfinal = Γi(ϕi.map(B∆.(mappings))) + Γ

jϕj (5.23)

There might need to be information, in B∆, with regard to earlier reduction-

with-authority, that performs synthesis.19 Equation 5.23 would modify to

Kfinal = (Γiϕi).map(B∆.(mappings)) + Γ

jϕj (5.24)

18Though it is not necessary to have sub-associations between individual folds and descriptionfragments, this was considered in Chapter 4 when folds φ represent parts of Φ - the deformedsurface.

19This might also counteract the possibility for the B∆ folds being an oversimplified vocabulary.


Transformation could continue as synthesis; applied to authoritative folds and/or

emerged situational folds, separately or collectively. Separate synthesis would have

the advantage of combining folds of similar confidence, but has to assume a linear

combination. Collective synthesis could provide a more creative interpretation but

with a lower confidence. In either case there would have to be a map to K-space,

which could possibly mean presenting B-space forms as K-space forms.

The accuracy, with which a new situation can be interpreted, is proportional to

the number of intermediate forms which can be recognized within tolerance. These

intermediate forms are the folds within the resultant surface, each of which will be

compared against folds found in B∆.

Tomography

In the tomographic domain, a situational dcalibseed is the seed scan (analogous to the

point in the series a radiologist might first look, though there is no guarantee that

they would be the same).

A radiologist would detect abnormal shapes and tissue densities by passing

forward and backward through the scans along an axis.

Moving to dseed+1 is moving forward; moving to dseed−1 is moving backward. To

recognize regions of (abnormal) tissue densities requires a recognition across several

scans, whereas individual emergence invocation only has interaction of one scan,

dseed. A 3D region of tissue is described by moving through the neighborhood.

dseed−α . . .dseed . . .dseed+β (5.25)

Adjacent tomographic sections are considered contiguous. Non-contiguous traver-

sal of (and invocation from) tomographic sections is dealt with in more detail in

Section 5.3.4.


If a single visual object data event is not strong enough to suggest a pathology

by itself, the radiologist can examine localized or extended regions to obtain a better

“feel” for the situation. This is also the case when they are looking to verify initial

or changing hypotheses. A good description, of a completely normal organ, can

be associated with a high confidence, despite not being of particular interest to the

recipient. There could be an assumption that the expert descriptions note everything

of importance including the items “normal” to most situations. Alternatively, there

could be an assumption of layered expert description. Different layers would have

caused different calibrations of the same situation of the calibration situations,

despite being aware of many more of the cues.

The calibration is assumed to use an existing corpus of data, and associated

descriptions. This is similar in notion to an MPEG data data stream and an

associated description stream.

The partitioning (or possibly overlapping maps onto sub-regions of the knowledge

space) of the expert descriptions deals with a mapping at a finer granularity or

specificity than the definitive data and expert description association.

A hypothetical ideal collection of data events (along an iteration path through

D), together with a traversal path (possibly backtracking), whose interaction mimics

the movement of the radiologist through the raw data while seeking the pathology

of the situation. More realistically there is the actual collection of data features used

in interaction by the framework; possibly missing cues.

Fold satisfaction events can be used for either augmentation of a hypothesis, or

modification of the confidence accompanying the hypothesis. However this needs

to happen as a natural consequence of the overall emergence; as part of the action

of the framework. The changing confidence, in the progressive interpretation of a


changing surface, reflects a gradual access to, and interaction of, more pertinent

data.

Layering of knowledge can reflect different purposes to data corpus description,

such as teaching and diagnosis. This is a special form of partial information. There

could have been calibration for each separately.

A fold is a region of a surface. A mappable fold matches a region of a B∆

tolerance surface; a B∆ fold . The emerged surface can contain other (possibly

overlapping) folds that partially match other (or the same) B∆ folds. In tomography

detail is important - for a less skilled observer than an expert; perhaps an assistant.

The detail must be such that different phenomena can be concurrently described.

Different data does not have to be separately described. The description is of the

significant referents; not of the data.

ϕi can be a fold that was created by the interaction of a data polyp (or a

concentration of polyps) in an organ. There will also be much detail which is

normal; perhaps corresponding to a different layer of calibration. The significance

of the presence of a polyp could be dependent on position within an organ (or a

body). The ‘section’ might need to correspond to a position or form ‘known’ by the

knowledge source or by the ‘experience’ stored in B∆.

Notions of malignant and benign growths possess (partially) a time varying

aspect that is not captured in a purely spatial modeling. The emergence with respect

to that aspect, is not based on the interaction of stateless data. It is possible that

future frameworks might explicitly allow some data input with respect to an earlier

description (or earlier data). Otherwise the onus is on the calling system or recipient

to use external reasoning to . The scope is restricted to phenomena that are purely

spatial in nature, or where some partial descriptions or cues in the description can

be purely spatial.


Γjϕj are the interaction results for which there do not exist mappings to corrected

K-space forms. To benefit from these, the recipient would either have to get a feeling

for surfaces, or the raw surfaces would need to be meaningful.

Some of the results ϕB0would pre-exist emergence from the domain description

B0. The initialization of B0 from K0 could provide some B-space to K-space map

information to B∆, in that an unmodified surface should be able to map back to

a domain description. It’s possible that some portions of Γjϕj will be sufficiently

similar to K-space form as to be recognisable by the recipient.

The results have lesser meaning or confidence in the absence of B∆ experience.

However, if there are multiple experience stores Bexpertise∆ , there could be different

interpreters or signifiers based on different sources of prior expertise. The ‘junk’ Γjϕj

in the context of one expertise source, could be meaningful in the context of another

source (which would require a separate Bδ).

Referent drift and confidence in some sense estimate how easily the unfamiliar

forms Γjϕj will be interpreted, as well as estimating how accurate/plausible the

internal intermediate representation is. There is a presentation issue for the calling

system, in deciding what to present. The order of presentation can be based on

confidence estimates from the framework.

Wildlife

In the wildlife domain there is no fixed nor approximately sized temporal extent

dseed−α . . . dseed+β, in D, which is analogous to the spatial extent, of a human

body, in the tomographic domain. A request for external selection from situation

data, would negate most of the benefit of machine filtering of situation data. An

arbitrarily or externally constrained temporal sub-sequence can specify the extents

of dseed.


With regard to calibration data, the descriptions of data by an expert can be

a parallel to a data stream (e.g. MPEG standards), and thereby the metadata

can provide the time-bounding. However, there would need to be several layers of

description, each with time-bounds where the description is true for the extent of

the time-bound. There will be descriptions relevant to each time-bound.

Leopards that behave unfamiliarly or animals that are leopard-like can expect

to influence the environment in ways both similar and different. In the case of

differently behaving leopards, the responses of the environment could be different;

uncertain deer rather than either calm or scattering. Deer just moving around

the water-hole should not cause perturbations that fall within tolerance of a fold

associated with ‘predation’ or ‘fear’ interpretations. Deer that scatter too much

might cause folds fail to be within tolerance for a scattering interpretation. The

result on the bridging entity will be that of folds shifting; some still within tolerance,

some outside. Birds would call warnings just at the proximity of the leopard.

Interactions with these sensory inputs stay within tolerance. Paw prints will not

change noticeably, but these are not picked up well by the sensors in any case.

In the case of a tiger, which is leopard-like the appearance is different, but will

be described much like a leopard. For the purpose of this situation, the system is

believed to be calibrated on leopards alone, whereas the environmental data which

it involves in emergence is that from a a tiger’s range. The situational description

is not a classification problem, so there won’t be simple labels of ‘leopard’ or ‘tiger’.

However the interpretation yielded by the system can be used in a classification

activity outside of the emergence. The system provides the filtered detections from

the sensor net to the recipient. The classification of ‘tiger’ rather than ‘leopard’ is

the recipient’s prerogative. The system’s task is to present an interesting situation

for perusal, with an accompanying description.


It is desirable for the nature of tolerance limits to allow for exaggerated events

that can be interpreted as a variant of a familiarly interpreted phenomenon. How-

ever, very exaggerated events can place the folds outside tolerance. This will be

neither deer wandering nor deer scattering, but something unfamiliar, and will be

presented to the recipient as such.

Implications

An issue is how much local knowledge would be needed for a description to be

useful. The usefulness of a description is dependent on the external purpose. There

might be much ‘description’ that has to be overlain on (or embedded in) original

data, or geographically positioned for the description to be useful. The form of

overlain description would be assumed useful (such as contour lines on a map).

Provision of a destination form (required for both current interpretation and prior

calibration) can include generalities of how different description types overlay or

embed. These different types or layers of description might also be generated by

separately calibrated layers of system experience, and held in different B∆ instances.

In polar geographic regions, where climate and the layering of ice keep shifting,

the experience of B∆ could be based on a context that rarely is repeated. The

reliability of tolerances of B∆ folds would be in turn authoritative for climatic and

positional tolerances. A topographic knowledge of the fixed landmasses might mean

that input data could be better able to ‘guess’ the extent and thicknesses of the

ice cover. However, there would be spatial limits for a given K0 that is used. Or

if K0 had global landmass information, the data might only be interacted with a

subset of the knowledge, or need a way for positional data to naturally become

salient. The knowledge for medical tomography would already possess an analog of

the fixed landmasses: ‘normal and familiar’ human anatomy. In the wildlife domain,


a framework based transformation system would encounter situations with different

terrain, much like the different landmasses. Terrain information is less critical in

the wildlife domain, if the calling system’s or recipient’s purpose is merely for more

observation opportunities.

5.3.4 Sparse Data

Multiple point emergence can occur from data events that are either contiguous

on non-contiguous. B∆ should not hold folds and bindings where the reduction,

from surface to folds, retained insufficient authority. Existing termination criteria

suggested by the initial model are the exceeding of an acceptance threshold con-

fidence Xaccept or a detection of negligible change in confidence δX negligible. These

termination criteria help the framework to reduce the amount of interaction that

needs to occur. However this risks premature termination and omission of descriptive

forms, if the acceptance threshold is set too high. The acceptance threshold can be

set by the calling system or derived from information in the bridging entity.

The calling system if acting as a filter for a knowledgable recipient could modify

the acceptance threshold for the framework, varying the interpretations presented to

the recipient. The framework does not encompass any feedback that occurs between

the calling system and the recipient. The framework is invoked by the calling system

whose decisions, with respect to framework parameters, might be affected by the

user. The data subset which (progressively) interacts with the bridging surface can

be affected by the emergence results. The calling system could request a complete

traversal, though this would increase the potential for referent drift.20

20And increased burden on the the calling system’s resources.


The domain knowledge within B0 or the situational experience within B∆ might

suggest relative spatial regions to invoke the next emergence in the iteration. The

path through the data might be affected by the progressive reference state.

Iteration can terminate if it runs out of data to interact with.

Tomography

The framework’s attempts to locate the data that caused triggered the descrip-

tion can look at the immediate neighborhood of a section. Alternatively the co-

occurrence of multiple abnormal fold events, stemming from a section along one

axis, can suggest promising sections along another axis. The system distinguishes

familiar/unfamiliar more so than normal/abnormal.

Spatial separation in a dimensional data space does not imply a corresponding

separation in the fold space or knowledge space. This situational discussion looks

at sparse evidence, which does not correspond to localized visual events. Some

phenomena will not be localized in all axes. The data has to be gathered from

greater extents of the organ. A tomographic section can contain non-contiguously

or unevenly distributed data events. These data events, unless part of a homogenous

distribution, will only occur in some perpendicular sections; sections along a different

scanning axis.

Distributed visual features can collectively suggest a pathology where individu-

ally they might not.

In figure 5.3 there are two polyps (data items of interest). They are situated far

enough apart to risk that iteration, along one axis (the dashed line), might terminate

before reaching both polyps. However, along another axis, they will appear in the

same scan/section.


Figure 5.3: Sparse events in non-contiguous sections

d0 Υ B0 : (. . .X0(polyp) . . .)=⇒B1 : (. . .X1(polyp) . . .) (5.26)

After the first interaction (equation 5.26), X1 > X0 indicates to the system

that the confidence in the presence of a polyp is higher than expected in a random

organ instance. If X1 < Xaccept then an interaction (equation 5.27) of B1 with the

neighboring environment (the data of section d1) occurs. d1 does not have to be

contiguous with d0

d1 Υ B1=⇒B2 : (. . .X2(polyp) . . .) (5.27)

The extent of neighborhood invocation from seed point d0 might not overlap with

alternative seed point d′

0 even if it overlaps the extent of neighborhood invocation

from d′

0. Supporting data, whose interaction would increase confidence, would not

be interacted.

A difference with a human expert is that the human expert would know what to

look for, and where to look for it. There is no guarantee that the knowledge in K0


will result in an appropriate selection of dk+1 where dk comprises the current data

section(s)21 .

If emergence occurs in parallel from different seed points, it will be as if the

system has access to future results, with respect to particular seed points. More

resources are used in less time. The time taken to combine emergence results, would

be similar to that of sequential emergence. Progressive descriptions can bleed faster

to the calling system.

There will still be degradation in confidence with the expanding neighborhood

for each seed section, but the cumulative evidence (for either acceptance (increase

in X )or refutation(decrease in X )) captured can be more comprehensive before the

time of termination. Calibration drift due to interaction with Bn for n > 0 is less-

ened. Though the possibility for overwhelming interaction might be increased (see

section 5.3.5). During traversal, the raw description confidence of fold population

confidences could change the acceptance threshold.

There are additional requirements for K0 and B∆. The importance of related

knowledge and data must be stored without the system becoming algorithmic or

prescriptive. Importance of knowledge can involve normality. Abnormality and

normality can be stated in K0 or learned during calibration.

Wildlife

The warning sounds of birds can be faint or not frequently repeated. Spatially co-

occurring data can co-occur across a wide temporal extent. Unlike the tomographic

data, the different forms and instances of indication, of leopards, can be spaced

well apart on the temporal axis. An exhaustive search might be needed at least

between the temporal points containing data that should indirectly interact in with

21In contiguous traversal of sections, there were multiple directions of expansion into theneighborhood, which implied multiple sections being used in subsequent interaction.


the bridging entity. Unfortunately, unlike the tomographic domain, there is not

necessarily finite restriction on the extents of an exhaustive neighborhood traversal

(aside from the totality of the data) or even a clear seed point from which to

arbitrarily select a traversal.

Casting the net too wide can introduce the issues found at the next discussed

level of complexity. The interacting data for a particular sub-referent can be tem-

porally disjoint. The temporally disjoint data for different sub-referents can be

interleaved. For example, two warning calls can be interleaved with unrelated

migratory movements of animals that are neither prey nor predator, in the context

of the warning calls. There can be cross-topic interference between the warning data

and the migratory data, while corroborating warning data has not yet co-occurred.

5.3.5 Detailed Situations

The last situation type, considers situations with lots of information. The salient

information of interest could be sparsely distributed, but the localities of interest

overlap, and collectively saturate D. Alternatively, the detail could be in concen-

tration (say of polyps in tomography) or numbers (say of animals in herds). It

considers the existence of multiple situations, each possibly indicated by many cues

from the data.

Tomography

The entire body contains anatomical information, any of which might be of interest

to the context of a given recipient. Though cross-topic interference is likely to be

a problem, the severity of the interference (depending on the pathology or purpose

of interest) can be of different degree in along different axes. This inconsistency of

degree of interference will be more pronounced the less ‘spherical’ the likely region


of interest within extents. More pronounced for entire body scans; less pronounced

for scans that are limited to organs.

The notion of concentration of polyps raises the question of how surfaces are per-

turbed. It’s not as if there can be external constraints applied to specific phenomena.

In some cases the concentration will be meaningful.

When looking at a patient’s data for the first time, if there is no existing

diagnosis hypothesis, a radiologist can be thought as looking at stateless information.

However, if tomographic data from a prior time or patient histories or other existing

hypotheses are known, they constitute context that provides state information.

That state information allows filtering of the patient data by the human medical

expert, which is not an option available to the framework. The notion of capturing

obscure cues does not philosophically allow for pre-processing of data, beyond any

pre-processing that would have occurred before the prior calibration.

Wildlife

There is a danger that too many instances of the same event will push the surface

deformation and the constituent folds beyond tolerance. For example, a partial

scattering within a migration of deer, might not be detected due to the seemingly

constant presence and movement of deer. Individual deer aren’t tracked as such,

though the motion of an individual does satisfy the notion of an almost subliminal

cue amongst other interacting cues (that over a large region would be providing a

constant description of migration). The wildlife analog of the tomographic section

is the entirety of the region containing sensors at one point in time. It does not have

the benefit of other obvious data paths or ‘concurrent’ information. If the feature

data is such that sectioning by point-in-time sectioning is used, motion of the deer

is an illusion or an inference that is not necessarily available. A deviation from one


form configuration to another could have meaning beyond the existence of either

configuration. However, tracking and associating form change (from source form

to destination form) is more complex than recognizing or associating form states.

The possibly infinite variety of beginning shapes mean that a greater range of form

changes can exist than actual possible forms (it would require a set of tolerances

held for each possible beginning form).

Some description is only possible via emergence across di, rather than emergence

within di. There is no clear data axis of iteration, for wildlife other than the passage

of an observer (human or framework) collecting or extracting “the data”. A concern

is whether a better than arbitrary iteration path can be obtained as a natural

consequence of the progressive emergence.

There can be more information in sensor catch than was experienced in the

calibration instances. The set of extra described sensor data (e.g. predator-prey)

can be indicative of a leopard chasing a deer. The sound events include the panicked

sounds of the deer, and sounds of movement of a leopard no longer intent on stealth.

It also includes the warning calls made by bird life. A richer-than-familiar predation

scenario can be described.

The passage of time through a sunset or sunrise could change the context of cues

that might only sometimes indicate predation. If some predators are nocturnal,

warning cues during the day might indicate the fear of prey animals, but not so

much the predation by predators.

Implications

The complexity of detailed data, in the wildlife domain, is manifest in the combina-

tion of different data references. This can involve climate, terrain, the interactions

of different species [JOW+02], and data of different media. There is a framework


assumption that all data elements can interact with the bridging surface. To miss

data sources, might be to lose non-linear effects. The result of all interactions

are a B-space form that are perturbations of the bridging surface. The referent

significance might be indicated by description of relationships and perhaps emotional

responses, such as fear.

5.4 Framework Implications for Situational Study

This last section summarizes the implications of the level-domain combinations,

and makes modifications to the framework. These implications and modifications

are used in the concluding chapter when future research avenues are suggested with

regard to the problem of improving situational references.

The complexity levels, of situations, from Section 5.3 were considered with regard

to the domains of tomography and wildlife.

“Familiar situations” shows the lack of guarantee of reproducible descriptions. A

combined nature for B∆ is required to provide tolerances for folds. If fold tolerances

are not inclusive of all instances of calibration situations, some calibration situations

will not be properly described. If the tolerances allow for all calibration instances,

this allows the possibility that a transformation system geared for “unfamiliar situ-

ations” will have multiple possible composed descriptions.

“Unfamiliar situations” involve folds from different experiences in B∆ that come

together to compose situational description (possibly in later synthesis after delivery

to the recipient). During calibration there might have been reduction of expert

description into sub-descriptions, resulting in multiple K-space interpretations for

a given fold. There could be a range of interpretations (represented in different

sub-regions of K-space) that emerge for a given situation. Though the destination


form can be specified, reference might be further improved through embedding in

or containment of data references (depending on domain).

“Partial familiarity” raises the possibility of layering calibration. This doesn’t

change the design of the framework. Rather it changes the usage of the framework,

with multiple instances of B∆ being used to handle differently extracted data feature

input. The different extraction would be either for different extractors or different

purposes, with a resultant descriptions all being in the same destination form. It

is important to note that there is no onus on the calling system to produce or find

different extractions. Calibration produces a separate B∆extractor for whatever data

extraction is available. This then allows each available B∆extractor to be used by the

framework-based transformation system to produce a K-space description of the

data.

“Sparse data” explores the issue of different iteration paths through the data D.

Depending on the data path, the reduction of D to a {. . .di . . .} could varyingly

separate data features (that need to interact) along the iteration path. This could

affect termination of iteration, and eventual interpretation of situations. With

respect to the tracking of change in confidence, there are issues with choice of

data path for sparse data. If termination occurs due to lack of appreciable change

|Xn+1−Xn| < Xsmall or lack of appreciable positive change (Xn+1−Xn) < Xsmall, not

all useful data might have been interacted. This connects to issues of non-contiguous

iteration, parallel emergence and alternative termination criteria.

“Detailed situations” raise the possibility of multiple events, possibly contribut-

ing to cross-topic interference. The richness of data can see premature termination

of iteration, by thresholding.

The framework offers alternative indicators (of similar complexity), to the recip-

ient, for situational interpretation by emergence. The eventual interpretations can


convey more information (about situational referents) to the recipients, that might

not be evident by their observations of the original situational data. This was the

basic notion behind lateral reference transformation.

Other aspects, of lateral reference transformation include:

• transformation of bridging entities by interaction with data;

• the selection of seed data portions;

• the effect on referent drift;

• familiar-unfamiliar vs normal-abnormal; and

• implied characteristics of prior calibration (of experience within B∆).

5.4.1 Transformation Characteristics

As the description references are more comprehensible than the data, it can seem

to the recipient, that the emergence transforms from a higher complexity to a lower

complexity.

For representations A and B, there is no notional difference between transforma-

tion A→B and B→A. There would have to be separate calibration of each directional

transformation, as there is no guarantee of inversion of either transformation.

To couple situational descriptions to raw data, the framework can at best provide

associations with the intermediate feature data input. If the path through the data

D is recorded, progressive description states can be correlated with the sequence of

di groupings used during iteration. The more parallel the interactions, the coarser

the granularity of data sources associated with situational descriptions, as more data

is involved in each situation. If the input data to the framework is different to the

raw data, there would be an onus upon the calling system to annotate source regions

of the raw data.


The arbitrary nature of an iteration path through a data space, implies difficulty

in consistently processing the same or similar data. The question is: does this

significantly affect the interpretation? If it does, a subsequent question arises: can

differently seeded hypotheses be significantly different, and still both be adequate?

The path through situational data, is a path feature data which might not correspond

to a path through raw data that an expert might follow.

Iteration of emergence invocation can occur across the knowledge space with

estimations of the corresponding spatial positions in the data space. The path of

traversal through the data neighborhood(s) will be affected by the mechanism of

interaction for that domain.

There can be a partial or progressive dispersal of description fragments to the

recipient, especially if layered calibration is used. Termination criteria provide an

initial point of possible termination. The recipient (or the calling system) can

decide whether to prolong the traversal. They should be informed that confidence

might drop, and confidence estimates should be presented along with progressive

descriptions. Progressive confidence levels can guide backtracking to earlier reference

states. The confidence levels can assist in estimating when referent drift becomes

too great.

Associating descriptions with emergence results, begs the question of appropriate

termination of interaction iteration. Resulting folds and tolerances are deemed to

indicate expert descriptions. The pertinent termination during calibration is that

of stoppage due to lack of change of confidence, rather than explicit threshold (to

avoid being dependent an an evolving threshold). Threshold is determined after

calibration interaction.

The threshold, for future situational interpretation, could be continuously mod-

ified, since any interaction is an incremental part of long-term experience. Though


the domain knowledge would still be considered static for the purposes of the

emergence.

5.4.2 Implied Characteristics of Prior Calibration

A calibrated implementation of the framework implied an adequate agreement in

interpretation between emerged descriptions and existing expert descriptions. As-

sociations exist with regard to matchable descriptive elements. The calibration of

B∆ was assumed to have been realized from sufficiently different instances, to lend

sufficient sophistication to fold granularity and recognition. Otherwise some folds

will have been either too coarsely mapped or mis-mapped. The requirement, that

the calibration-data corpus is constructed independently of the framework, should

be maintained.

Fold matching within tolerance

Familiar folds can emerge from the interactions of unfamiliar data and the bridging

entity. If all of the folds are recognizable within tolerance, each fold can be mapped

to an existing description fragment, which is a structural or representational ref-

erence element. This representational element is of a form easily understood by a

recipient of the system. An overall description is composed by the synthesis of these

fragments, to the stage at which the recipient is better at analysis.

Some situational data sets which interact with the knowledge contain more than

one situation. Whereas an expert description, corresponding to a calibration in-

stance, might describe only one situation or one aspect of a situation. This is possibly

addressed by the usage of layered calibration (multiple B∆). Different experts can

comment on different indications picked up from a scene, and be influenced by their

own contexts.


If only some folds ϕj (within the emerged fold collection Φ) are within known

tolerances, these folds might have to contain information from implicit cues from the

situational data. The implicit cues will need to have taken part in interaction during

calibration. If the requisite information is spatially spread or otherwise subliminal

within the interacting data set, the calibration iterations needs must have picked

up the implicit cues. It is important that most of the calibration data be included

in transformation. Iteration should not terminate early, during transformation of

calibration data references. Though the extents, of what is considered exhaustive

data, will depend on the domain.

Effect on Referent Drift

There were two proposed alternative termination criteria for iteration: acceptable

confidence above a threshold, and negligible difference in confidence. The tracking of

the change in confidence seemed more appealing as it could have occurred during cal-

ibration as well. Whereas, a threshold value would either have to be determined after

calibration, or set by the calling system. The latter might enable the adjustment

of iteration with respect to external resource usage. The implemented framework

will consume resources that are provided by the calling system. The threshold

confidence could be altered with respect to resource usage, or set according to a

resource threshold. Determining threshold after calibration, is more acceptable in

the constrained situation of the thesis. Though with regard to the bigger picture

and dynamic experience, the threshold might keep changing as the domain is better

understood.

However, tolerances are also by-products of the folding from many calibration

instances. Initial arbitrary confidence thresholds can be modified until acceptable

descriptions emerge.


K0 exists, and describes some of what is possible in the domain. It is unchanging

during emergence of situational descriptions. System experience with data features

and any calibration is stored in B∆. B∆ is tied into use with the framework, whereas

the construction of K0 is not. The difference between K0 and B∆ is in the associated

situational data for each parcel of B∆ description. Whatever interaction is possible,

between bridging surfaces and data features, is involved in a transformation system

following the framework.

5.4.3 Framework Modification

The basic framework is sound, though there can be a change of approach toward

two aspects of the framework. Sparse data benefits from alternative paths of data

traversal. The introduction, of layered calibration, addresses the use of experience

stores that are individually based on incomplete expert description.

Path of Data Traversal

Iteration works better with domains that naturally decompose into partitions. There

is also benefit to domains for which there are convenient seed points.

However, even if the domains provide data with natural decomposition, the

choice of alternate paths, along which to traverse data neighborhoods, can im-

prove the effectiveness of the reference transformation. Some domains, such as

tomography, come with convenient multiple paths improve the chances of concurrent

interaction of salient elements. Though other domains, such as wildlife observation,

have less clear choice of path. Depending on the domain, the selection of seed data

position is not guaranteed to be intuitive. Application of the iterative aspect of

the framework requires a seed point (a body section position, the time of sensor

set capture). Tomography, naturally partitions itself into sub-parts, where the


partitioning corresponds to the organization of the raw data. There exist ‘centre’ and

‘end’ sections, which are good candidates for seeding, if traversal is along contiguous

sections.

The chosen, possibly arbitrary, paths of the observer provide context in the sense

of influence that is not described. It is assumed, for most cases, that the path made

through the media affects the interpretation. Just as experienced humans can make

different interpretations, it is acceptable for an automation to attain a plausible

interpretation instead of a definitive one. However, the framework should work to

minimize the bias of the path.

Layered Calibration

Though it does not change the basic nature of the framework, the inclusion of

multiple experience stores can improve the effectiveness of the framework. Each

experience store, B∆, represents a particular calibration based on familiar data

with incomplete descriptions. Collectively, multiple experience stores can provide

different opinions with regard to situational interpretation.

Chapter 6

Conclusion

The work of the thesis is the investigation, primarily by means of a transformation

framework, of lateral reference transformation. The existence of stored knowledge

and experience is assumed. Context is considered from several perspectives. From

the perspective of the data, it is the knowledge and the experience available for

interpretation. This includes entities and concepts that aren’t directly observable

in the data. From the perspective of the description, context is whatever influences

transformation and subsequent interpretation, without being explicitly indicated

by the description. For the purposes of emergence and interpretation, the context

of the recipient would be the ideal context, but not guaranteed as obtainable. The

context of an expert, as implicit in the stored experience, is considered an acceptable

context.

During transformation, situational data references (in one form of indication)

are replaced by description references (in another form of indication). Though data

(D) and description (Kfinal) are represented in different forms, each indicates a

situation. The situation is a referent or is comprised of several referents. Though,

ideally the same situation, some referent drift is expected. The framework has been

designed with imperfect transformation in mind. In this study, it is a bridging

196

CHAPTER 6. CONCLUSION 197

entity corresponding to the destination form that is modified. The bridging entity

is initially a guess derived from the domain description (K0) represented in the

destination form. There is transformation of indication from one reference form to

another. The scope is limited to a designated input referential form and a designated

output referential form. There exists a calibrated experience store B∆ in addition to

the initial bridging entity B0. The initial reference is comprised of what is normally

referred to as data; typically a collection of captured and possibly processed data.

The resultant reference is a designated output form, known as a description in the

sense that it is more easily interpretable by a recipient.

6.1 Contributions

The principal contribution of this thesis is the framework for transforming refer-

ences. It is based on the notion that reference transformation can occur between

representations of similar complexity. The framework is designed such that any

prior calibration could have been performed on associated data and descriptions

that weren’t specifically generated for use with the framework. The framework

adopts external representation as the basis of internal representations known as

bridging entities. These interact, and are perturbed by data references. This

also enables the data references indirectly interact with each other. Viewing the

perturbed entities as surfaces enables references to be modeled as a collection of folds

that can be mapped to representations in the destination form. Confidence in the

resultant representations can be based on tolerances specified in prior calibration.

The flexible intermediate representation enables the framework to provide lateral

reference transformation, while maintaining domain independence.


6.1.1 Flexible Intermediate Entities and Domain Indepen-

dence

Emergence cannot normally be relied upon to generate a desired representation.

The approach with reference transformation, is to begin with internal entity (based

the domain knowledge), and to perturb it through interaction with the data. The

domain knowledge (in the destination form) is itself not considered an internal

entity, as it is available outside of the framework, and would have been generated

independently of the framework.

There are two main instances of internal entity; both based on the same internal

representation. The internal representation, is based on the domain knowledge as

represented in the destination form. One entity is the initial bridging entity B0,

which is used to transform the data references of each new situation. The other

entity is the experience store B∆, which holds calibration information based on the

prior transformation of data references from all expert described situations, be they

normal or abnormal. The calibration is assumed to have occurred during invoked

transformation of familiar, described, situational data and bridging entities.

It is the ability to adopt the destination form, as the basis of intermediate

representation, that frees the framework from dependence on specific forms of rep-

resentation (so long as the bridging entities and the situational data are able to

interact).

The flexibility in choice of internal representation (bridging entity) also helps

to maintain domain independence. External representations, that are used with

domains, are adopted and slightly augmented for use as internal bridging entities.

Corresponding external experience is also adopted by creating an auxiliary entity

for use as an experience. Though the creation of an experience store requires a

pre-processing step, to calibrate the behavior of bridging entities (such as surfaces),


upon interaction with data. Using the experience, with reference transformation of

data references to familiar situational phenomena, assists in the interpretation of

the transformation of data references to new situations.

6.1.2 Lateral Reference Transformation

An emphasis on clarity rather than complexity, allows lateral reference transforma-

tion to provide alternative situational indicators for use in situational interpretation.

Reference transformation is invoked during early emergence, to modify the elements

taking part in later emergence.

The conventional notion of emergence has interactions of elements at a lower

level of complexity interacting to create a system of higher complexity. A greater

sophistication is often seen in the greater complexity. Lateral reference transforma-

tion focuses on an increase in clarity. While a change in complexity might also occur,

there is no requirement for attainment of higher complexity. A semblance of greater

resultant sophistication lies in clearer indication of the reality of the situation. The

improvement is in communicated information; not absolute information. Easier

comprehension of the situation, might lead the recipient to the perception that

complexity has lessened.If the referent drift is kept low, referent complexity will also

be maintained.

More flexible emergence is facilitated by the lateral transformation of data ref-

erences. However, the framework assumes that the transformation is imperfect. It

is designed to internally self-correct to converge the description referent to the data

referent.

A data reference (D) is to be transformed into a description based on an initial

source of domain knowledge (K0). A source of knowledge B0 (derived from K0) is

used as the initial approximation of all situational referents. The domain knowledge


RDRBn

RKfinal

D

K0 B0

Bn Kfinal

Figure 6.1: Framework for Lateral Reference Transformation

is deemed to not change, for the purpose of trusting the emerged description. A

entity B∆ stores framework experience for the purposes of interpreting the resultant

emerged final bridging entity Bn. Information in B∆, from the original map from

K0 to B0, allows mapping back to K-space in the form of Kfinal. Ideally, the

description referent (R (Kfinal)) will not have drifted appreciably from the data

referent (R (D)). Confidence estimates are used, as the referents cannot be measured

directly.

Some phenomena will fall outside both the familiar-normal and the familiar-

abnormal. The unfamiliar-normal and the unfamiliar-abnormal might be partially

described similarly to the familiar as some elements of the situations are likely to

be common.

The confidence of interpretation will be higher for familiar situations, as the

results of interaction with familiar data are more likely to fall within tolerance

than the results of interaction with new data. Some captured data corresponding

to unfamiliar phenomena will interact and create folds that are within calibration

tolerances. The mapping from surface folds to description fragments will not deal

with some emerged folds.


The mapped description will most likely not map every result of the interaction

of important data. The variety of folds, that develop during the progressive per-

turbation of the intermediate bridging entity Bn, are possibly infinite in number

depending on the flexibility of knowledge representation.

The transformed description is created after iterated interactions of situational

data with progressive perturbations of the bridging entity. The termination of

iteration is based on confidence estimates associated with the latest reference state.

If X (R (ρn) = R (ρ0)) > Xthreshold then iteration can terminate with confidence

deemed acceptable. Iteration can also terminate if there is no appreciable gain in

confidence.

The framework does not guarantee that there will be indication of some aspect

of the situation, that the recipient does not infer from the data. Trivially this can be

the case if there is no indicator that an expert could have absorbed from the data or

if the description is complete. However, complete descriptions cannot be presented

with complete confidence.

The framework improves the ability of implicit cues to be made explicit. Verifi-

cation via increase in confidence helps to highlight the new description portions that

are relevant. During iteration, intermediate forms with varying amounts of implicit

and explicit information are possible. Anything indicated by the data, which is not

described by the existing knowledge source, is considered to be part of the implicit

knowledge. Any knowledge added by emergence, which didn’t already exist in the

knowledge source, is considered to be an explicit statement of previously unknown

phenomena. This is ‘unknown by the system experience’. After augmentation of

the knowledge source, that knowledge is considered implicit in by the use of the

knowledge source. The destination reference form is what makes the information

explicit to the client (person or machine) that uses it.


The reference transformation framework can work in concert with verification

techniques. For example, being interleaved with relevance feedback and user ob-

servation. However, the benefit of the framework is for offline/machine-only pro-

cessing/transformation of references. Appropriate verification is more in the line of

consulting an ontology. Situational ontologies, if built, will allow for this.

The framework takes advantage of existing natural reduction within data to real-

ize smaller data portions, which take part in emergence interactions with knowledge.

Emergence occurs several times during iteration of invoked transformation. After

transformation to the destination form, the descriptions can be used in continuing

emergence, which might in turn involve synthesis of the descriptions.

Calibration will have been performed per feature extractor, per expert. This

enables individual customizations based both feature extractors and experts to be

modular. The framework allows for the possibility for layered experience, where

different stores of experience (B∆)expertContext can be used to provide interpretations

of different elements of the situation. The current iterative checking of interpre-

tation hypotheses employs as yet unused, though already available data portions.

Information on unused data needs tracking by the framework or calling system. A

layered approach to calibration enables the emergence of further descriptions that

can be appreciated together by a recipient. One layer’s interpretation can provide

corroboration for another layer’s interpretation.

6.2 Further Work

6.2.1 Requesting Further Evidence

Interpreting for a wider audience involves the sharing of many experts’ experience

or opinion, and allows greater focus on the application of experience. One calibrated


experience store Balternative∆ can be used to test the hypotheses emerging from the

use of another experience store Boriginal∆ . Verification, by alternatively generated

description, would need to occur naturally as part of the emergence.

The notion of progressive data interaction partially envisions future research

which can request new data transformation to verify hypotheses based on existing

data features. This can involve revisiting data regions whose data have already

been interacted, or possibly the entirety of D. The data will be processed in several

passes, as verification is sought in addition to several passes for existing layers.

While emergence is principally considered to be from data to knowledge, knowl-

edge to data can be useful, especially if supporting evidence is sought. Consider the

sub-structures of a knowledge source to be raw units, and the feature data to be the

knowledge representation. Different extractors lose different details. It is difficult to

name a particular data need, and then choose an arbitrary extractor.

6.2.2 Augmentation of Domain Knowledge and Experience

The only learning, that has been discussed, in Chapter 4, is learning with regard

to the currently indicated situation. This is manifest in the descriptions, but not

as a lasting modification of K0. K0 holds domain knowledge; B∆ holds domain

experience.

The point of description is to impart a better understanding of the world,

including the notion of normality, where the normality is what is most expected.

For the most part, new knowledge will be gathered slowly, in that the ratio of

the change in knowledge to original knowledge will be small. What is more likely

is that advances in technology will change the available input features, perhaps

necessitating complete recalibration.


Each iteration of emergence invocation further perturbs the bridging entity Bn.

If B∆ was calibrated by interacting calibration data with B0, the interpretation

would be more likely to be valid in earlier iterations. Though there can be iterative

deformation of B0 by emergence during calibration, the iterative deformation of B0

during interpretation can follow different paths. This can be seen as a drift from

calibration, increasing the likelihood of referent drift as well.

A concern about the validity of calibration, might limit modification of K0 to

add-only rather than update. The impact of the drift from calibration is also a

lessening of confidence (that the original referent has been maintained).

Prior experience associated with data is re-modeled with the current scene or

situation. The current situation can add to the understanding of both the current

situation and the overall domain.

For augmentation of knowledge and experience, a time varying aspect could

be introduced. Some initial knowledge source would have to be existing or boot-

strapped. A K0 changing with time, K0(t), would raise the issue of trustworthiness

of prior calibration. A system’s knowledge can be grown from interpretations of

new data. For learning K0(t)⋃

β =⇒K0(t+1), where K0(t+1) is the modified

knowledge base after learning, and β is the acquired knowledge after emergence.

Accreted experience has to persist over all runs, whether it originated from

calibration, bootstrapping or new situations. The fusion of new knowledge and

experience is a possibly separate issue to the creation, depending on the means of

creating K0.

If the knowledge source is permanently modified, this can have non-trivial effects

on an interpretation (including earlier interpretation). New interpretations of the

same data might be different, if invoked after a knowledge source is modified.

The point-in-time knowledge during an activity of emergence, provides part of the


context. So it can be argued that the earlier interpretation was made for a different

context, even if the data has not changed.

The recipient can choose to add the data and emerged description (or custom

description) to B∆. Tolerances might be altered if the description of the new data

is the same as for prior situations. Notionally, changing tolerances for forms can

validate or invalidate interpretations of the past. However, this is also the case for

human experience. The experience base is improved for future interpretation.

References

[AAYA08] Besma R. Abidi, Nash R. Aragam, Yi Yao, and Mongi A. Abidi. Survey

and analysis of multimodal sensor planning and integration for wide

area surveillance. ACM Computing Surveys, 41(1):7:1–7:36, December

2008.

[ABBS00] Bryan Adams, Cynthia Brezeal, Rodney. A. Brooks, and Brian

Scassellati. Humanoid robots, a new kind of tool. IEEE Intelligent

Systems, 15(4):25–31, July 2000.

[ACMH03] Karl Aberer, Philippe Cudre-Mauroux, and Manfred Hauswirth. The

chatty web: Emergent semantics through gossiping. In WWW2003,

pages 197–206, Budapest, Hungary, 2003.

[AF08] Sabah Al-Fedaghi. Modeling communication: One more piece falling

into place. In The 26th ACM International Conference on Design of

Communication, pages 103–110, September 2008.

[AL03] John Arena and Pamela Lipson. Another point of

view in machine vision. Test and Measurement World,

http://www.tmworld.com/article/CA279582.html, March 2003.

Last accessed 23 June 2009.

[AW05] Edward Altman and L Wyse. Emergent semantics from media

blending. In Managing Multimedia Semantics, pages 363–390. Idea

Group, 2005.

[BCQ+07] Cristiana Bolchini, Carlo A. Curino, Elisa Quintarelli, Fabio A.

Schreiber, and Letizia Tanca. A data-oriented survey of context models.

SIGMOD Record, 36(4):19–26, December 2007.

[BDBS08] Marco Bertini, Alberto. Del Bimbo, and Giuseppe Serra. Learning

ontology rules for semantic video annotation. In MS 2008 second

international workshop on the many faces of multimedia semantics,

pages 1–8, 2008.

206

REFERENCES 207

[BDBT06] Marco Bertini, Alberto Del Bimbo, and Carlo Torniai. Automatic

annotation and semantic retrieval of video sequences using multimedia

ontologies. In ACM Multimedia 2006, pages 679–682, October 2006.

[BDFF+08] S. Biasotti, L. De Floriani, B. Falcidieno, P. Frosini, D. Giorgi,

C. Landi, L. Papaleo, and M. Spagnuolo. Describing shapes by

geometrical-topological properties of real functions. ACM Computing

Surveys, 40(4):12:1–87, October 2008.

[BHB+07] M. Betke, D.E. Hirsh, A. Bagchi, N.C. Hristov, N.I. Makris, and T.H.

Kunz. Tracking large variable numbers of objects in clutter. In IEEE

Conference on Computer Vision and Pattern Recognition, 2007, pages

1–8, June 2007.

[Bit01] Maria Bittner. Surface composition as bridging. Journal of Semantics,

18:127–177, 2001.

[BL04] Ronald J Brachman and Hector J Levesque. Knowledge Representation

and Reasoning. Morgan Kaufmann, San Francisco, CA 94111, 2004.

[Bor01] Luigi Borzacchini. Light as a metaphor of science: A pre-established

disharmony. Semiotica, 2001(136):151–171, 2001.

[BQR08] Cristiana Bolchini, Elisa Quintarelli, and Rosalba Rossato. Relational

Data Tailoring Through View Composition, volume 4801 of Lecture

Notes in Computer Science, pages 149–164. Springer Berlin /

Heidelberg, 2008.

[Bre06] Cynthia Breazeal. Human-robot partnership. IEEE Intelligent

Systems, pages 79–81, July/August 2006.

[BT00] Eric Bonabeau and Guy Theraulaz. Swarm smarts. Scientific

American, 282(3):54–61, 2000.

[Cho02] Noam Chomsky. Syntactic Structures. Walter de Gruyter, 2nd edition,

2002.

[Chu90] Tsai Chih Chung. The Book of Zen. Asiapac Books, 1990.

[CKB08] Andy Cockburn, Amy Karlson, and Benjamin B Bederson. A review

of overview + detail, zooming and focus + context interfaces. ACM

Computing Surveys, 41(1):2:1–2:31, December 2008.

[CR06] Aaron Ceglar and John F. Roddick. Association mining. ACM

Computing Surveys, 38(2):1–42, July 2006.

REFERENCES 208

[Cro05] James L. Crowley. Situated observation of human activity. In CVIIE

’05: Proceedings of the Computer Vision for Interactive and Intelligent

Environment, pages 97–108, 2005.

[Cru93] James P Crutchfield. The calculi of emergence. In Complex Systems

- from Complex Dynamics to Artificial Reality, pages 1–54, Numazu,

Japan, 1993.

[Cru94] James P Crutchfield. Is anything ever new? - considering emergence.

In G Cowan, D Pines, and D Melzner, editors, Integrative Themes,

volume XIX of Sata Fe Institute Studies in the Sciences of Complexity,

pages 479–497. Addison-Wesley, Reading MA, 1994.

[DA00] James C. Duncan and Nicholas Ayache. Medical image analysis:

Progress over two decades and the challenges ahead. IEEE

Transactions on Pattern Analysis and Machine Intelligence, 22(1):85–

106, 2000.

[Dav03] Marc Davis. Theoretical foundations for experiential systems design.

In ETP ’03: Proceedings of the 2003 ACM SIGMM workshop on

Experiential telepresence, pages 45–52, Berkeley, CA, USA, 2003.

[DBLI07] F. Deloule, D. Beauchene, P. Lambert, and B. Ionescu. Data fusion

for the management of multimedia documents. In 10th International

Conference on Information Fusion, pages 1–7, July 2007.

[DDN03] Marc Davis, Chitra Dorai, and Frank Nack. Understanding media

semantics. Tutorial notes: ACM Multimedia 2003, Nov 2 2003.

[DGD08] Stefan Dietze, Alessio Gugliotta, and John Domingue. Towards

context-aware semantic web service discovery through conceptual

situation spaces. In CSSSIA ’08: Proceedings of the 2008 international

workshop on Context enabled source and service selection, integration

and adaptation, pages 1–8. ACM, 2008.

[Doa02] An Hai Doan. Learning to Map between Structured Representations of

Data. PhD thesis, University of Washington, 2002.

[Dub08] DublinCore. Dublin core metadata element set, version 1.1.

http://dublincore.org/documents/dces/, 2008. Last accessed 24 June

2009.

[EPKM+07] S. Espinosa Peraldi, A. Kaya, S. Melzer, R. Moller, and M. Wessel.

Toward a media interpretation framework for the semantic web. In

REFERENCES 209

IEEE/WIC/ACM International Conference on Web Intelligence, pages

374–380, 2007.

[ES03] E A Engbers and A W M Smeulders. Design considerations for

generic grouping in vision. IEEE Transactions on Pattern Analysis

and Machine Intelligence, 25(4):445–457, 2003.

[FGLH03] Jianping Fan, Yuli Gao, Hangzao Luo, and Mohand-

Said Hacid. A novel framework for semantic image

classification and benchmark. In SIGKDD Workshop

on Multimedia Data Mining, Washington, August 2003.

http://www.springerlink.com/content/p568l74775106641/fulltext.pdf


[FRLK08] Simone Fuchs, Stefan Rass, Bernhard Lamprecht, and Kyandoghere

Kayamakya. A model for ontology-based scene description for context

aware driver assistance systems. In Proceedings of the 1st International

Conference on Ambient Media and Systems, pages 1–8, February 2008.

[GD97] John S Gero and Lan Ding. Learning emergent style using an

evolutionary approach. In B Verma and X Yao, editors, ICCIMA,

pages 171–175, Australia, 1997. Griffith University.

[Gri09] Patrick Grim. Semantic content and pragmatic convention: Emergence

through individual advantage in spatialized environments. In IEEE

Symposium on Artificial Life, pages 37–44, March 2009.

[Gru93] Thomas R Gruber. A translation approach to portable ontology

specifications. Knowledge Acquisition, 5(2):199–220, 1993.

[GS00] Amarnath Gupta and Simone Santini. Toward feature algebras in

visual databases: The case for a histogram algebra. In IFIP Working

Conference on Visual Databases (VDB5), pages 248–259, Fukuoka,

Japan, 2000.

[GSF02] William I Grosky, D V Sreenath, and Farshad Fotouhi. Emergent

semantics and the multimedia semantic web. SIGMOD Record,

31(4):54–58, 2002.

[GYWA05] Jonathon M E Gabbai, Hajun Yin, W A Wright, and Nigel M Allinson.

Self-organization, emergence and multi-agent systems. In International

Conference on Neural Networks and Brain, pages 1858–1863, 2005.

REFERENCES 210

[Har99] Robert L Harris. Information Graphics: a comprehensive illustrated

reference. Oxford University Press, 1999.

[Hil05] Diane Hillmann. Using dublin core. Dublin Core Metadata Initiative,

http://dublincore.org/documents/usageguide/, 2005. Last Accessed 28

June 2009.

[HL08] Mark J. Huiskes and Michael S. Lew. Performance evaluation of

relevance feedback methods. In CIVR ’08: Proceedings of the 2008

international conference on Content-based image and video retrieval,

pages 239–248, New York, NY, USA, 2008. ACM.

[HM08] Markus. C. Huebscher and Julie. A. McCann. A survey of autonomic

computing - degrees, models, and application. ACM Computing

Surveys, 40(3):7:1 – 7:28, August 2008.

[Hol00] J H Holland. Emergence: from chaos to order. Oxford University

Press, Oxford, 1st edition, 2000.

[Hoo01] Hoogs. Multi-modal fusion for video understanding. In 30th Applied

Imagery Pattern Recognition Workshop, pages 103–108, Washington,

D.C., 2001.

[HSH+08] James. Hendler, Nigel. Shadbolt, Wendy. Hall, Tim. Berners-lee, and

Daniel. Weitzner. Web science: An interdisciplinary approach to

understanding the web. Communications of the ACM, 51(7):60–69,

July 2008.

[Hun03] Jane Hunter. Enhancing the semantic interoperability of multimedia

through a core ontology. IEEE Transactions of Circuits and Systems

for Video Technology, 13(1):49–58, January 2003.

[Hun08] Jane Hunter. Tracking the progress of multimedia semantics - from

mpeg-7 to web3.0. In SAMT ’08: Proceedings of the 3rd International

Conference on Semantic and Digital Media Technologies, pages 1–?,

2008.

[Jai03a] Ramesh Jain. Experiential computing. Communications of the ACM,

46(7):48–55, 2003.

[Jai03b] Ramesh Jain. Folk computing. Communications of the ACM, 46(3):27–

29, 2003.

REFERENCES 211

[JOW+02] Philo Juang, Hidekazu Oki, Yong Wang, Margaret Martonosi,

Li Shiuan Peh, and Daniel Rubenstein. Energy-efficient computing

for wildlife tracking: design tradeoffs and early experiences with

zebranet. In ASPLOS-X: Proceedings of the 10th international

conference on Architectural support for programming languages and

operating systems, pages 96–107, New York, NY, USA, 2002.

[Kan06] Makoto Kanzaki. Insulin receptor signals regulating glut4 translocation

and actin dynamics. Endocrine Journal, 53(3):267–293, 2006.

[Kat91] S. D. Katz. Film directing shot by shot: visualizing from concept to

screen. Michael Wiese Productions, 1991.

[Ker02] Andruid Kerne. Concept-context-design: A creative model for the

development of interactivity. In Creativity and Cognition, pages 192–

199, Loughborough, UK, 2002. ACM Press.

[KHL+07] Akrivi Katifori, Constantin Halatsis, George Lepouras, Costas

Vassilakis, and Eugenia Giannopoulou. Ontology visualization

methods—a survey. ACM Computing Surveys, 39(4):10:1–10:43,

October 2007.

[KJ08] Jeffrey A. Kleim and Theresa A. Jones. Principles of experience-

dependent neural plasticity: Implications for rehabilitation after

brain damage. Journal of Speech, Language and hearing Research,

51(Supplement):S225–S239, February 2008.

[KKS+07] Andruid Kerne, Eunyee Koh, Steven M. Smith, Hyun Choi, Ross

Graeber, and Andrew Webb. Promoting emergence in information

discovery by representing collections with composition. In ACM

Creativity and Cognition, pages 117–126, June 2007.

[KS96] Zoltan Kovecses and Peter Szabo. Idioms: A view from cognitive

semantics. Applied Linguistics, 17(3):326–355, 1996.

[KS06] Daisuke Kitayama and Kazutoshi Sumiya. An evaluation system for

news video streams and blogs. In Proceedings of the 2006 ACM

Symposium on Applied Computing, pages 1361–1368, 2006.

[KS07] Markus Koskela and Alan F Smeaton. An empirical study of inter-

conept similarities in multimedia ontologies. In ACM International

Conference on Image and Video Retrieval, pages 464–471, July 2007.

REFERENCES 212

[KTF02] A Kojima, T Tamura, and K Fukunaga. Natural language description

of human activities from video images based on concept hierarchy of

actions. International Journal of Computer Vision, 150(2):171–184,

2002.

[Kui00] Benjamin J Kuipers. The spatial semantic hierarchy. Artificial

Intelligence, 119:191–233, 2000.

[Lan90] Chris Langton. Computation at the edge of chaos: Phase transitions

and emergent computation. Physica D, 42(1-3):12–37, 1990.

[Lan01] Adam Langley. Freenet. In Andy Oram, editor, Peer-to-Peer:

Harnessing the Benefits of a Disruptive Technology, pages 123–132.

O’Reilly, Sebastopol, CA, 2001.

[LB07] Jeff Lieberman and Cynthia Breazeal. Tikl: Development of a wearable

vibrotactile feedback suit for improved human motor learning. IEEE

Transactions on Robotics,, 23(5):919–926, October 2007.

[LG09] Chi-Tat Law and Joshua I Gold. Reinforcement learning can account

for associative and perceptual learning on a visual-decision task. Nature

Neuroscience, 12(5):655–663, May 2009.

[LPR98] H Levesque, F Pirri, and R Reiter. Foundations for the situation

calculus. Linkoping Electronic Articles in Computer and Information

Science, http://www.ep.liu.se/ea/cis/1998/018/, 1998. Last Accessed

28 June 2009.

[LSD+06] Michael S Lew, Nicu Sebe, Chabane Djeraba, LIFL, and Ramesh

Jain. Content-based multimedia information retrieval: State of the

art and challenges. ACM Transactions on Multimedia Computing,

Communications and Applications, 2(1):1–19, February 2006.

[MA01] Douglas L Medin and Cynthia Aguilar. Categorization. In The MIT

Encyclopedia of the Cognitive Sciences, pages 104–105. MIT Press,

2001.

[Mae02] Maedche. Emergent semantics for ontologies. IEEE Intelligent Systems,

17(1):85–86, 2002.

[McL01] Brian P McLaughlin. Emergentism. In The MIT Encyclopedia of the

Cognitive Sciences, pages 267–269. MIT Press, 2001.

[Min88] Marvin Lee Minsky. The Society of Mind. Touchstone (Simon and

Schuster), New York, 1st edition, 1988.

REFERENCES 213

[MMB+09] Remington J. Moll, Joshua J. Millspaugh, Jeff Beringer, Joel Sartwell,

Zhihai He, Jay A. Eggert, and Xiwen Zhao. A terrestrial animal-

borne video system for large mammals. Computers and Electronics

in Agriculture, 66(2):133–139, 2009.

[MP05] Rokia Missaoui and Roman M Palenichka. Effective image and video

mining: an overview of model-based approaches. In Proceedings of the

6th international workshop on Multimedia data mining, pages 43–52,

August 2005.

[MS04] C. Muller-Schloer. Organic computing - on the feasibility of controlled

emergence. In International Conference on Hardware/Software

Codesign and System Synthesis, pages 2–5, 2004.

[MY08] Abdul-Rahman Mowlood-Yunis. Fault-tolerant semantic mappings

among heterogeneous and distributed local ontologies. In Proceeding of

the 2nd international workshop on Ontologies and nformation systems

for the semantic web, pages 31–38, October 2008.

[Nac02] Frank Nack. The future of media computing. In Svetha Venkatesh

and Chitra Dorai, editors, Media Computing, pages 159–196. Kluwer,

Boston, 2002.

[Nin09] Ana Nino. Machine translation in foreign language learning: language

learners’ and tutors’ perceptions of its advantages and disadvantages.

ReCALL, 21(2):241–258, 2009.

[NK97] Y Nakamura and T Kanade. Spotting by association in news video.

In Fifth ACM International Multimedia Conference, pages 393–401,

November, 1997.

[NLF07] Eduardo. F. Nakamura, Antonio. A. Loureiro, and Alejandro. C. Frery.

Information fusion for wireless sensor networks: Methods, models and

classifications. ACM Computing Surveys, 39(3):9:1–9:55, August 2007.

[Ove09] Paul Over. Trec video retrieval evaluation. National Institute of Stan-

dards and Education, http://www-nlpir.nist.gov/projects/trecvid/,

2009. Last accessed 22 June 2009.

[Per07] Fernando Pereira. Mpeg multimedia standards: Evolution and future

developments. In ACM Multimedia 07, pages 8–9, 2007.

REFERENCES 214

[PKK03] Valery. A. Petrushin, Anne Kao, and Latifur Khan. The 4th

international workshop on multimedia data mining. SIGKDD

Explorations, 6(1):106–108, 2003.

[Pol97] Michael Polanyi. The tacit dimension. In Knowledge in Organizations,

pages 135–146. Butterworth-Heinemann, 1997.

[PPT08] Antonio. Penta, Antonio. Picariello, and Letizia. Tanca. Multimedia

knowledge management using ontologies. In MS 2008: second

international workshop on the many faces of multimedia semantics,

pages 24–31, October 2008.

[PRB+07] Thierry Pun, Patrick Roth, Guido Bologna, Konstantinos Moustakas,

and Dmitri Tzovaras. Image and video processing for visually

handicapped people. EURASIP Journal on Image and Video

Processing, 2007:1–12, 2007.

[QHR+07] Guo-Jun Qi, Xian-Sheng Hua, Yong Rui, Jinhui Tang, Tao Mei, and

Hong-Jiang Zhang. Correlative multi-label video annotation. In ACM

Multimedia 07, pages 17–26, September 2007.

[QQL06] Dacheng Qu, Hong Qu, and Yushu Liu. Emergence in swarming

pervasive computing and chaos analysis. In 1st International

Symposium on Pervasive Computing and Applications, pages 83–89,

August 2006.

[Qui06] Joel Quinqueton. Emergence in problem solving, classification

and machine learning. In Proceedings of the Eighth International

Symposium on Symbolic and Numeric Algorithms for Scientific

Computing, pages 5–9, September 2006.

[RBVdS07] Marko A. Rodriguez, Johan Bollen, and Herbert Van de Sompel. A

practical ontology for the large-scale modeling of scholarly artifacts and

their usage. In JCDL ’07: Proceedings of the 7th ACM/IEEE-CS joint

conference on Digital libraries, pages 278–287. ACM, June 2007.

[Rei01] Daniel Reisberg. Learning. In The MIT Encyclopedia of the Cognitive

Sciences, pages 460–461. MIT Press, 2001.

[Rei09] Marga Reimer. Reference. Stanford Encyclopedia of Philosophy,

http://plato.stanford.edu/entries/reference/, May 2009. Last accessed

26 June 2009.

REFERENCES 215

[Rey87] Craig Reynolds. Flocks, herds, and schools: A distributed behavioral

model. Computer Graphics, 21(4):25–34, 1987.

[RK03] Victor W. Rodwell and Peter J. Kennelly. Proteins: Higher orders of

structure. In Harper’s Illustrated Biochemistry, pages 30–39. McGraw

Hill, 2003.

[RSLC08] John F. Roddick, Myra Spiliopoulou, Daniel Lister, and Aaron Ceglar.

Higher order mining. SIGKDD Explorations Newsletter, 10(1):5–17,

2008.

[RSN03] A. Viranga Ratnaike, Bala Srinivasan, and Surya Nepal. Making

sense of video content. In Proceedings of the 11th ACM International

Conference on Multimedia, pages 650–651, 2003.

[RSN05] A. Viranga Ratnaike, Bala Srinivasan, and Surya Nepal. Emergent

semantics: An overview. In Managing Multimedia Semantics, pages

351–362. Idea Group, 2005.

[Rus00] D Russell. A design pattern-based video summarization technique. In

Proceedings of the 33rd Hawaii International Conference on System

Sciences, pages 1–5, 2000.

[Sam01] Jerry Samet. Nativism, history of. In The MIT Encyclopedia of the

Cognitive Sciences, pages 586–587. The MIT Press, 2001.

[Sau83] Ferdinand de Saussure. Course in General Linguistics. McGraw-Hill,

New York, 1983.

[SBC07] A Scherp, S Boll, and H Cremer. Emergent semantics in personalized

multimedia content. Journal of Digital Information Management, 5(2),

April 2007. http://medien.informatik.uni-oldenburg.de/pubs/2006-

ScherpBollCremer-SemanticMM4U.pdf, Last Accessed 28 June 2009.

[SD08] Simone Santini and Alexandra Dumitrescu. Context as a non-

ontological determinant of semantics. In SAMT ’08: Proceedings

of the 3rd International Conference on Semantic and Digital Media

Technologies, pages 121–136, 2008.

[SGJ01] Simone Santini, Amarnath Gupta, and Ramesh Jain. Emergent

semantics through interaction in image databases. IEEE Transactions

on Knowledge and Data Engineering, 13(3):337–351, 2001.

REFERENCES 216

[SGS08] Carsten Saathoff, Marcin Grzegorzek, and Steffen Staab. Labelling

image regions using wavelet features and spatial prototypes. In SAMT

’08: Proceedings of the 3rd International Conference on Semantic and

Digital Media Technologies, pages 89–104, 2008.

[SJ99] Simone Santini and Ramesh Jain. Interfaces for emergent semantics in

multimedia databases. In SPIE, Storage and Retrieval for Image and

Video Databases, pages 167–175, San Jose, USA, 1999.

[SJ07] Ansgar Scherp and Ramesh Jain. Towards an ecosystem for semantics.

In ACM MS’07, pages 3–11, September 2007.

[SK07] Lambrini Seremeti and Achilles Kameas. Multimedia ontologies. In

Proceedings of the 3rd international conference on Mobile multimedia

communications, pages 1–7, August 2007.

[SKW07] Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. Yago: A

core of semantic knowledge unifying wordnet and wikipedia. In 16th

International World Wide Web Conference, pages 697–706, May 2007.

[SM83] G. Salton and M. J. McGill. Introduction to Modern Information

Retrieval. McGraw-Hill, New York, 1983.

[SNN+08] Ansgar Scherp, Frank Nack, Klara Nahrstedt, Masashi Inoue, Andreas

Girgensohn, Andreas Henrich, Philipp Sandhaus, Sabine Thieme, and

Michelle Zhou. Interaction and user experiences with multimedia

technologies - challenges and future topics. In ACM HCC’08 3rd

international workshop on human-centered computing, pages 1–6,

October 2008.

[Sos86] A.B. Sossinsky. Tolerance space theory and some applications. Acta

Applicandae Mathematicae, 5:137–167, 1986.

[SPT06] Susan Stepney, Fiona A. C. Polack, and Heather R. Turner.

Engineering emergence. In Proceedings of the 11th IEEE International

Conference on Engineering of Complex Computer Systems, pages 89–

97, 2006.

[SSFNY08] Eri Sato-Shimokawara, Yusuke Fukusato, Jun Nakazato, and Toru

Yamaguchi. Context-dependent human-robot interaction using

indicating motion via virtual-city interface. In IEEE International

Conference on Fuzzy Systems, pages 1922–1927, June 2008.

REFERENCES 217

[SSR03] Harini Sridharan, Hari Sundaram, and Thanassis Rikasis. Computa-

tional models for experiences in the arts, and multimedia. In ACM

Multimedia 2003, 1st ACM Workshop on Experiential Teleprescence,

Berkeley, CA, USA, 2003.

[Sta02] S Staab. Emergent semantics. IEEE Intelligent Systems, 17(1):78–79,

2002.

[Swi88] Virginia M Swisher. Similarities and differences between spoken

languages and natural sign languages. Applied Linguistics, 9(4):343–

356, 1988.

[SWS+00] Arnold W. M. Smeulders, Marcel Worring, Simone Santini, Amarnath

Gupta, and Ramesh Jain. Content-based image retrieval at the end of

the early years. IEEE Trans. Pattern Anal. Mach. Intell., 22(12):1349–

1380, December 2000.

[SWS05] Cees. G.M. Snoek, Marcel. Worring, and Arnold W.M Smeulders. Early

versus late fusion in semantic video analysis. In ACM Multimedia 05,

pages 399–402, November 2005.

[SZ00] Simeon J Simoff and Osmar R Zaiane. Report on mdm/kdd2000:

The 1st international workshop on multimedia data mining. SIGKDD

Explorations, 2(2):103–105, December 2000.

[TAC+08] Christian Timmerer, Maria Teresa Andrade, Pedro Carvalho, Davide

Rogai, and Giovanni Cordara. The semantics of mpeg-21 digital items

revisited. In ACM MS’08, pages 17–23, October 2008.

[TD08] Ruben Tous and Jaime Delgado. Semantic-driven multimedia retrieval

with the mpeg query format. In SAMT ’08: Proceedings of the 3rd

International Conference on Semantic and Digital Media Technologies,

pages 149–163, 2008.

[TMKW07] Matthew E. Taylor, Cynthia Matuszek, Bryan Klimt, and Michael

Witbrock. Autonomous classification of knowledge into an ontology.

In The 20th Annual FLAIRS Conference, pages 140–145. AAAI Press,

May 2007.

[TvdM01] Jochen Triesch and Christoph von der Malsburg. Democratic

integration: Self organized integration of adaptive cues. Neural

Computation, 13:2049–2074, 2001.

REFERENCES 218

[Ult07] Alfred Ultsch. Emergence in self organizing feature maps.

In Proceedings of the 6th International Workshop on Self-

Organizing Maps, pages 1–7, August 2007. www.uni-

marburg.de/fb12/datenbionik/pdf/pubs/2007/WSOM07Emergence


[UW08] Chris Urmson and William Whittaker. Self-driving cars and the

urbanchallenge. IEEE Intelligent Systems, pages 66–68, March/April

2008.

[VD01] Svetha Venkatesh and Chitra Dorai. Computational media aesthetics:

Finding meaning beautiful. IEEE Multimedia, pages 10–12, 2001.

[Vio01] Patrizia Violi. Semiotics and cognition. In The MIT Encyclopedia of

the Cognitive Sciences, pages 744–745. MIT Press, 2001.

[Wal92] M M Waldrop. Life at the Edge of Chaos. Touchstone (Simon and

Schuster), New York, 1992.

[Wan08] Lihong V Wang. Tutorial on photoacoustic microscopy and computed

tomography. IEEE Journal of Selected Topics in Quantum Electronics,

14(1):171–179, January/February 2008.

[WF05] Ian H Witten and Eibe Frank. Data Mining: Practical Machine

Learning Tools and Techniques. Data Management Systems. Morgan

Kaufmann, 2nd edition, 2005.

[WG02] George M Whitesides and Bartosz Grzybowski. Self-assembly at all

scales. Science, 295(5564):2418–2421, March 2002.

[WJ07] Utz Westermann and Ramesh Jain. Toward a common event model

for multimedia applications. IEEE MultiMedia, 14(1):19–29, Jan/Mar

2007.

[WN08] Xiao-Yong Wei and Chong-Wah Ngo. Fusing semantics, observability,

reliability and diversity of concept detectors for video search. In ACM

Multimedia 08, pages 81–90, October 2008.

[WZ08] Zhen. Wen and Michelle. X. Zhou. An optimization-based approach to

dynamic data transformation for smart visualization. In ACM IUI’08,

pages 70–79, January 2008.

[WZZ08] Changhu Wang, Lei Zhang, and Hong-Jiang Zhang. Learning to reduce

the semantic gap in web image retrieval and annotation. In SIGIR ’08:

REFERENCES 219

Proceedings of the 31st annual international ACM SIGIR conference

on Research and development in information retrieval, pages 355–362,

July 2008.

[YCKH07] A. Yanagawa, S.-F Chang, L. Kennedy, and W. Hsu. Columbia

university’s baseline detectors for 374 lscom semantic visual concepts.

Technical Report 222-2006-8, Columbia University, March 2007.

[YXL+03] Xinguo Yu, Changsheng Xu, Hon Wai Leong, Qi Tian, Qing Tang,

and Kong Wah Wan. Trajectory-based ball detection and tracking

with applications to semantic analysis of broadcast soccer video. In

ACM Multimedia 03, pages 11–20, November 2003.

[ZG02] Rong Zhao and William I. Grosky. Narrowing the semantic gap -

improved text-based web document retrieval using visual features.

IEEE Transactions on Multimedia, 4(2):189–200, June 2002.

[ZM08] Jack Zhao and Andrew Vande Moere. Embodiment in data sculpture:

a model of the physical visualization of information. In DIMEA ’08:

Proceedings of the 3rd international conference on Digital Interactive

Media in Entertainment and Arts, pages 343–350, 2008.

[ZPRH07] Zhihong Zeng, Maja Pantic, Glenn I Roisman, and Thomas S Huang.

A survey of affect recognition methods: Audio, visual and spontaneous

expressions. In The Ninth International Conference on Multimodal

Interfaces, pages 126–133, November 2007.

[ZZM+08] Xiang Sean Zhou, Sonja Zillner, Manuel Moeller, Yiqiang Zhan, Arun

Krishnan, and Alok Gupta. Semantics and cbir: A medical imaging

perspective. In ACM CIVR’08, pages 571–580, July 2008.

ERRATA

• p 6 para 3, 7th line: replace “A lines of paint” with “A line of paint”

• p 7 para 3, 6th line: replace “being used an element” with “being used asan element”

• p 13 para 1, 2nd line: delete “be of to”

• p 14 para 2, 4th line: replace “of similarly complexity” with “of similarcomplexity”

• p 14 para 4, 5th line: delete “for”

• p 14 para 6, 2nd line: delete “a”

• p 20 para 2, 6th line: delete “be”

• p 21 para 1, 2nd line: delete “language”

• p 21 para 2, 4th line: insert a period “.” after “to comprehend [DGD08]”

• p 22 para 3, 7th line: insert a space “ ” between “MPEG-21” and “is a”

• p 30 para 1, 3rd line: delete “that”

• p 44 para 4, 1st line: replace “is not” with “are not”

• p 44 para 4, 2nd line: delete “for whom the available”

• p 46 para 3, 4th line: replace “be possible” with “been possible”

• p 47 para 2, 1st line: replace “presents the a theory” with “presents atheory”

• p 66 para 2, 3rd and 4th lines: replace “can be of a representational that”with “can be a reference whose form”

• p 75 para 3, 2nd line: replace “one criterion” with “One criterion”

• p 79 para 2, 2nd line: replace “attempts produce” with attempts to pro-duce”

• p 79 para 3, 2nd line: replace “consistent trend is toward” with “consistenttrend toward”

• p 82 para 2, 7th line: replace “B∆ is stores” with “B∆ stores”

• p 93 para 4, 2nd line: replace “Allowing synthesis” with “Synthesis is”

• p 94 para 4, 1st line: replace “If Bn+1 is comprised” with “Bn+1 can becomprised”

• p 103 para 1, 1st line: replace “candidate description possess” with “can-didate descriptions possess”

• p 110 para 2, 1st line: replace “then the initial” with “then by the initial”

• p 110 para 3, 4th line: replace “referent. if either” with “referent. Ifeither”

• p 114 para 3, 6th line: replace “proportion of of” with “proportion of”

• p 122 para 1, 3rd line: replace “kδ is a unfamiliar phenomena” with “kδ

is an unfamiliar phenomenon”

• p 128 para 2, 3rd line: replace “looking at a situations” with “looking atsituations”

• p 130 para 6, 1st line: replace “it will up to” with “it will be up to”

• p 131 para 3, 3rd line: replace “criterion could the first” with “criterioncould be the first”

• p 146 para 2, 4th line: replace “is a desirable” with “is desirable”

• p 146 para 3, 3rd line: delete “that”

• p 147 para 2, 7th line: delete “to which”

• p 157 para 3, 4th line: replace “Could get the same start (seed) time” with“The same start (seed) time can be obtained”

• p 158 para 4, 7th line: replace “An unfamiliar situations” with “An unfa-

miliar situation”

• p 159 para 1, 1st and 2nd lines: replace “Finally, a complex situation

allow for indicators to lie across extents of data, overlap and possiblyindicate multiple concurrent phenomena.” with “Finally, a complex situa-

tion allows for indicators to overlap, lie across extents of data, or indicatemultiple concurrent phenomena.”

• p 164 para 1, 1st line: replace “will provided information” with “willprovide information”.

• p 171 para 3, last line: replace “isolation of the deer as individuals arealready be” with “isolation of the deer, as individuals, is already”

• p 182 para 3, 1st and 2nd lines: replace “The framework’s attempts tolocate the data that caused triggered the description can look at the im-mediate neighbourhood of the section.” with “As with other domains,progressive data selection can occur within an immediate neighbourhood;passing along axially adjacent sections.”

• p 187 para 1, 1st line: replace “form” with “form of”

• p 191 para 4, 5th line: replace “dependent an an” to “dependent on an”

lateral reference transformation for emergenceusers.monash.edu/~srini/theses/viranga_thesis.pdf ·...

Documents