machine perception and creativity building on image schema conceptual metaphor and blending by...

Upload: zaheer-ahmad

Post on 05-Apr-2018

220 views

Category:

Documents


0 download

TRANSCRIPT

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    1/76

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    2/76

    Agenda

    Research Question New AI, Cognition ( Imagery, Embodiement )

    Some Theory About

    Image Schema, Conceptual Metaphor and Conceptual

    Blending Computational models of image schemas and

    reasoning Gestalt Principle

    Nash Equilibrium Research Proposal

    Building on: Concepts to Think and Reason Creatively

    Emotions, Reasoning and Image Schema---an idea

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    3/76

    Research Question

    How to develop complex concepts by mapping

    simple ones onto each other, to let the computerPerceive, think and reason creatively

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    4/76

    From New AI to GOFAI, A Flashback

    Second AI winter 19871993 In the 80s and 90s, many cognitive scientists and AI researchers also

    rejected the symbol processing model of the mind and argued that thebody was essential for reasoning, a theory called the embodied mindthesis

    Another encouraging event in the early 1980s was the revival of

    connectionism in the work of John Hopfield and David Rumelhart, fromthis AI had achieved success but not for too long

    Dates back to 1956 when John McCarthy of MIT invited many leadingresearchers of the time to a workshop where he introduced the termartificial intelligence.

    Among the participants were Marvin Minsky , Herbert Simon, and AllanNewell, the founding fathers, so to speak, ofartificial intelligence

    What originated from this meeting, and what came to be the guidingprinciples until the mid-1980s, was what is now known as the classical,symbol-processing paradigm, also known as the cognitivistic paradigm

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    5/76

    Cognitive Linguistics

    In linguistics, cognitive linguistics (CL) refers to the branch oflinguistics that interprets language in terms of the concepts,

    sometimes universal, sometimes specific to a particular

    tongue, which underlie its forms.

    It is thus closely associated with semantics

    cognitive linguists view meaning in terms ofconceptualization

    Aspects of cognition that are of interest to cognitive linguists

    include:

    Construction grammar and cognitive grammar. Conceptual metaphor and conceptual blending.

    Image schemas and...

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    6/76

    Thought is impossible without an image.

    Aristotle, On Memory and Recollection

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    7/76

    Cognitive Linguistics and Imagery

    Imagery is visualizing, seeing in the mind's eye, hearing

    in the head , imagining the feel of,

    thought to be caused by the presence of picture-like

    representations (mental images) in the mind / brain.

    Over the course of the past 2 decades, imagery has once again

    become increasingly more interesting to cognitive scientists.

    A number of studies have shown that humans automatically

    and unconsciously engage perceptual and motor imagery when

    performing high-level cognitive tasks ( hearing, speaking,seeing)

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    8/76

    Cognitive Linguistics and Imagery

    Human beings, rationality is embodied

    That is their thinking is in visible form

    containing one inside another.

    two types of imaginative structures are

    presented to explain the embodied

    experience functions in our life,

    image schema and

    Conceptual metaphorical projection

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    9/76

    Nouvelle AI and Embodied Cognition Philosophers, cognitive scientists and artificial

    intelligence researchers who study embodiedcognition and the embodied mind believe that

    the nature of the human mind is largely determined by the

    form of the human body.

    They argue that all aspects of cognition, such as ideas,thoughts, concepts and categories are shaped by aspects

    of the body.

    These aspects include the perceptual system, the

    intuitions that underlie the ability to move, activities andinteractions with our environment and the native

    understanding of the world that is built into the body and

    the brain.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    10/76

    Johnson argues that : Meaning cannot be separated from the structures

    of our embodied perceptual interactions andmovements. If we did not have the bodies we do,

    or if they were somehow radically different thanthey are now, then we would not create,understand, and communicate meaning in theway we do.

    Meanings in natural language come with figurative(metaphor) and multivalentpatterns that cannot be reduced to literal concepts and propositions;

    the patterns and their connections are embodied and cannot be reduced to literalconcepts and propositions.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    11/76

    The embodied mind thesis is opposed to other theories of cognitionsuch as cognitivisim, computationalismand Cartesian dualism.

    George Lakoff(a cognitive scientist and linguist) and hiscollaborators (including Mark Johnson, Mark Turner, and Rafael E.Nez) have written a series of books promoting and expanding thethesis based on discoveries in cognitive science, such as conceptualmetaphor and image schema.

    Robotics researchers such as Rodney Brooks, Hans Moravec andRolf Pfeifer have argued that true artificial intelligence can only beachieved by machines that have sensory and motor skills and areconnected to the world through a body.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    12/76

    Definitions as Reminder of Journey

    Cartesian dualism----I think, therefore I am

    Cognitivisim/Symbolism---symbols and logic , Words,

    Sentence, Discourse, Pragmatics Connectionism---NN-----

    Embodiement -----Image schema---conceptualmetaphor---blending

    Statistical/ Empirical --Corpora other stats machinelearning tech..

    Hybrid Approaches

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    13/76

    Embodiement asStrong AI / AGI (may be)

    Embodiement is generally presented as Strong AI

    Models/Simulation of actual thinking, is called weak AI,

    Programs that are actually thinkingis known as the strong AI

    weak AI position is unproblematic and generally accepted:

    The nature of the simulation model is clearly different from the thing it simulatesas in a simulation of rain the computer does not get wet, the model of thinking isdifferent from the thinking process itself.

    It is the strong AI stance with which people often take issue.

    This is not surprising. It is unsettling for many people to believe that a computer isactually thinking, rather than just simulating the process.

    the classical paradigm has had its definite successes, but it has failed to make clearthe nature of intelligence

    Where as Emobodied Cognition has shown some potentials in the area of StrongAI

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    14/76

    Image Schema

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    15/76

    Linguistic interpretation of image schema

    In Cognitive Linguistics, the term image schema implicatesperception in all acts of conceptualization. Concepts develop

    from representation of a perceptual conglomeration(merging)

    of multiple senses, including visual, auditory, haptic, motoric,

    olfactory, and gustatory(taste) experiences.

    Schemas are structures of the imagination [Kant]

    schemas are fixed templates superimposed onto perceptions

    and conceptions to render meaningful representations *

    Image Schema is a structures that allow us to store and

    categorize our basic physical and sensory experiences in

    different meaningful ways**

    *Towards Computational Model of Image Schema **The Body in the Mind(1990)

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    16/76

    Image Schema Image schemas are relatively simple structures that constantly

    recur in our everyday bodily experience: CONTAINERS, PATHS,LINKS, FORCES, BALANCE ,and in various orientations and

    relations:

    UP-DOWN, FRONT-BACK, PART-WHOLE, CENTER-PERIPHERY *

    According to Johnson, image schemas are structures thatorganize our mental representations at a level more general and

    abstract.

    An image schema is a recurring, dynamic pattern of our

    perceptual interactions and motor programs that gives

    coherence and structure to our experience

    It is universally agreed that image schemas are mental patterns

    associated with broad classes of concepts or experiences**

    **Perception to Meaning by Beat Hamper*Lakoff 1987: 267, emphasis added

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    17/76

    different kinds of image schemas are

    containment (such as houses, cars, and boxes);

    force (which causes a door to stay open or results

    from contact with others);

    up-down orientation (which arises because we

    live in a gravitational field)

    structure of our conceptual schemais primarilydetermined by practical criteria, rather than

    abstract or logical ones*

    Embodied Cognition: A field guide, by Andrson

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    18/76

    Evidence of Image Schema

    Existence

    Neuroscientists have found evidence indicatesactivation patterns like image schemas exist in

    both animal and human brains

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    19/76

    How Image Schemas are Formed

    Based on our physical features, we create concepts asright and left, front and back, near and far and ashuman beings able to walk, so we create concepts assource, path, and goal*, all these are image schemas

    On account of being confronted by forces that can pullor push us (wind, animals, and other human beings) wecreated concepts as compulsion, attraction andblockage of movement.

    As a consequence of our orthostatic upright position,we incorporate concepts as balance and verticality.

    *Learning phrasal verbs through image schemas: a new approach by Sarah Barbieri Vieira

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    20/76

    The main image schemas are:

    SOURCE-PATH-GOAL

    CONTAINER

    LINK BALANCE

    PART-WHOLE

    CENTER-PERIPHERY

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    21/76

    Extensions to Image Schema

    Some linguists have added new image schemas to theoriginal list proposed by its forerunners. E.g.

    The BOUNDED REGION

    If this region has only one dimension, we have the

    SOURCE-PATH-GOAL schema. If it has two dimensions, we have the SURFACE schema.

    Finally, if it has three dimensions, we have the CONTAINERschema.

    A room, for instance, would be the prototypical example of

    a CONTAINER, but a table would be also a CONTAINER. According to Peas framework, a table would be a

    SURFACE.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    22/76

    From the CONTAINER image we can raise twosubsidiary images: FULL-EMPTY and EXCESS as in sentences

    She is full of confidence or She has an excess of

    confidence. Linked to the SOURCE-PATH-GOAL schema, we

    can get the FRONT-BACK image as in a sentence I think we need to step backfrom this situation.

    Linked to the SURFACE schema, we can get still NEAR-FAR and CONTACT.

    As shown in the following diagram

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    23/76

    We have placed some image schemas under SURFACE/SOURCE-

    PATH-GOAL because they are subsidiary to both of them.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    24/76

    Typical Image Schema

    An image schema will have parts and relations.

    The parts represent a set of entities, and therelations represent the connection amongdifferent parts, like

    causal relations, temporal sequences, part-wholepatterns, etc. Figure 1 can be thought of as a PATHschema.

    It contains three parts, a source point A, aterminal point B, and a trace between them.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    25/76

    The basic meaning of this schema can beunderstood as moving from A to B in aphysical environment. i.e. representing a

    physical connection.. and

    It can be used to understand the sentence like

    the melting of ice into water,

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    26/76

    The IN-OUT schema shown in figure below.

    The Schema has two parts as landmark (LM)and trajectory(TR). This schema clearlyrepresents the visual situation of the firstsentence.

    1. Mary got out of the car.Here the circle represents the car, and Mary moves along thearrow out of the car. Since the car cant be circular, Mary

    cant move along a straight line in leaving the car, so thisschema gives us only one idealized image.

    In-Out Schema

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    27/76

    Other Image Schemas

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    28/76

    From the above examples, we can see that imageschemas are dynamic patterns rather than fixedand static images, as their visual diagramsrepresent them.

    They are dynamic in two important respects:1. schemas are structures of an activity by which

    we organize our experience in ways that we cancomprehend;

    2. unlike templates, schemas are flexible in thatthey can take on any number of specificinstantiations in varying contexts

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    29/76

    Properties of image schemas

    Image schemas are pre-conceptual in origin.

    However once the recurrent patterns of sensory information

    have been extracted and stored as an image schema, sensory

    experience gives rise to a conceptual representation.

    This means that image schemas are concepts,

    but of a special kind: they are the foundations of the

    conceptual system, because they are the first concepts to

    emerge in the human mind, image schemas are so fundamental to our way of thinking that we are not consciously aware of them, we take our awareness of what it

    means to be a physical being in a physical world very much for granted because we acquire this knowledge so early in life, certainlybefore the emergence of language.

    Image schema transformations:

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    30/76

    Image schema transformations:

    the body within the brain Image schemas do not simply exist as single entities, rather

    We can perform mental operations on image schemas that

    are analogs of spatial operations. For example,

    we can rotate mental image both in 2D and 3D situation, and

    that also at a fixed rate of approximately 60 degree per

    second, it indicates primary schematic operations, called

    image-schema transformations (i.e. IS are linked together to

    form natural relationships)

    Image schema transformations have been shown to play a

    special role in linking perception and reason. (Lakoff 1987)

    image schemas motivate important aspects of how we think,

    reason, and imagine [cognitive linguisticsGeeraerts Al Eds

    2006]

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    31/76

    Conceptual Metaphor

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    32/76

    Metaphorical Projection

    Metaphorical interpretation is the bridge by which wecan understand and structure one domain of

    experience in terms of another domain of a different

    kind.

    Metaphor is thus defined both as a linguisticphenomenon in which vocabulary is shared among

    different domains, and as a conceptual one in which

    different conceptual domains are linked by metaphoric

    mappings

    Image Schema has thus shows to lie at the basis of

    numerous metaphorical construction(mapping)*

    * Metaphor We Live By

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    33/76

    Metaphorical projection, allows us to project

    patterns from one domain of experience into

    the structure of a different domain

    Here, metaphor is not only a linguistic mode

    of expression; rather, it is one of the chief

    cognitive structures by which we are able to

    have coherent, ordered experiences that wecan reason about and make sense of

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    34/76

    Conceptual Metaphors*

    Conceptual metaphor defines an understanding of one conceptual domain

    in terms of another; for example, using one persons experience of life to

    understand a different persons experience.

    A conceptual domain can be any coherently represented experience.

    Conceptual domains play one of the following two roles in conceptual

    metaphors:

    (a) Source domain: the conceptual domain from which we draw

    metaphorical expressions

    (b) Target domain: the conceptual domain that we try to understand or

    explain

    Metaphorical expressions present themselves in the form of TARGET is

    the SOURCE where target is the concept to be characterized by the

    mapping of constituent elements from the source concept.

    Metaphor A Practical Introduction

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    35/76

    Model for metaphor learning

    When we are exposed to a new idea or concept, the brain

    tries to understand it.

    we try to ground the concept by building some mental models

    to capture it.

    This model may be built either directly in its own right or as

    composed of other models and symbols.

    Direct grounding is likely for spatial or temporal phenomenon

    only.

    Image schemas such as PATH, SUPPORT, CONTAINMENT etc,

    have been proposed to model this basic understanding of the

    physical world. More complex and abstract domains like

    nance, relationships, politics etc need to be grounded in

    terms of other grounded symbols.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    36/76

    examples can demonstrate the importance of

    metaphorical projection in helping people to

    connect knowledge from different domains.

    More is up is the most typical one.

    More is up implies that we understand

    quantity in terms of a verticality schema in our

    everyday experience. Examples like

    Prices keep going up

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    37/76

    and Turn down the heat suggest that we

    understand MORE as being oriented UPWARD.

    The two schemas MORE and UP are therefore

    correlated in our experience that provides a

    physical basis for our abstract understanding

    of quantity.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    38/76

    Current Computational models of

    image schemas and reasoning One explicit attempt to model image schemas can be found from

    Regiers work [1996].

    Regier focused on spatial relations concepts, like in, out, from, to,on, off, front, back, above, below. Like. the book is on/above/below/in front.

    He used a connectionist network and the following input:

    1. project on a screen a set of movies consisting of a sequence ofpictures of a geometrical figure either stationary or movingrelative to others;

    2. pair each movie with a spatial relation term in some naturallanguage that correctly characterizes the spatial relation depictedon the input screen.

    3. Given such pairs of movies and terms, Regier has the networklearn spatial relations concepts

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    39/76

    Differentiation of Conceptual and

    Linguistic Metaphor Through Example

    Here color-capitals are used for the statement ofconceptual metaphors and italics for metaphoricallinguistic expressions.

    AN ARUGMENT IS A WAR

    Your claims are indefensible.

    He attacked every weak pointin my argument.

    His criticisms were right on target.

    I demolishedhis argument.

    Ive never won an argument with him.

    You disagree? Okay, shoot!

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    40/76

    If you use that strategy, hell wipe you out. He shot down all of my arguments.

    LOVE IS A JOURNEY

    Look how farweve come.

    Were at a crossroads. Well just have to go our separate ways.

    We cant turn backnow.

    I dont think this relationship is going anywhere.

    Where are we?

    Were stuck.

    Its been a long, bumpy road.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    41/76

    This relationship is a dead-end street. Were just spinning our wheels.

    Our marriage is on the rocks.

    Weve gotten off the track.

    This relationship isfoundering.

    THEORIES ARE BUILDING

    Is that thefoundation for your theory?

    The theory needs more support.

    We need to constructa strong argument for that.

    We need to buttress the theory with solid arguments.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    42/76

    the linguistic expressions (i.e., ways of talking)make explicit, or are manifestations of, theconceptual metaphors (i.e., ways of thinking).

    To put the same thing differently, it is themetaphorical linguistic expressions that revealthe existence of the conceptual metaphors.

    The terminology of a source domain that is

    utilized in the metaphorical process is one kind ofevidence for the existence of conceptualmetaphor

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    43/76

    Mapping of Metaphors

    There is a set of systematic correspondences between thesource and the target in the sense that constituent conceptual

    elements ofb correspond to constituent elements ofa.

    Technically, these conceptual correspondences are often

    referred to as mappings.

    Let us look at some cases where elements of the source domain

    are mapped onto elements of the target domain.

    Lets take the love is a journey conceptual metaphor first. When

    we use the sentence We arent going anywhere,the expressiongo somewhere indicates traveling to a destination, in this

    particular sentence, a journey which has no clear destination.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    44/76

    The word we obviously refers to the travelers involved.

    This sentence then gives us three constituent elementsof journeys:

    the travelers, the travel or the journey as such,and thedestination.

    The relationship is foundering suggests that somehowrelationships are conceptually equated with thevehicles used in journeys.

    The sentence Its been a bumpy road is not about thephysical obstacles on the way but about the difficultiesthat the lovers experience in their relationship.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    45/76

    Metaphor Mapping

    Consider the reversed conceptual metaphors mapping,source- target (concrete to the more abstract concept)

    to make it easily Understandable

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    46/76

    Christmas is two days away

    (Time is Space)

    two days away recasts anevent (Christmas) as a

    location with respect to thespeakers current location

    in time by specifying a

    temporal interval (twodays) as a distance*

    Fig shows: Conceptcorrespondence between source

    and target domains

    Metaphor Mapping Graphically

    *Towards a cognitive model of conceptual blending by Markus, School of Informatics,

    University of Edinburgh

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    47/76

    Metaphor as Knowledge

    A significant amount (perhaps the vast

    majority) of what cognitive psychologists

    consider knowledge resides in the human

    mind as a collection of conceptual metaphorsthat are built up and categorized over years of

    experience*

    * Lakoff

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    48/76

    Metaphor, Reason and Imagination*

    The reason we have focused so much on

    metaphor is that it unites reason and

    imagination. Reason, at the very least, involves

    categorization, entailment, and inference. Imagination, in one of its many aspects, involves

    seeing one kind of thing in terms of another kind

    of thingwhat we have called metaphorical

    thought. Metaphor is thus imaginative rationality

    *Metaphor We Live by

    I i ti d R i

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    49/76

    Imagination and Reasoning

    Johnson concurs with the nineteenth-century view that

    imagination connotes creativity, invention and artistic

    expression.

    However, he makes the case that imagination also plays

    a critical role in human reasoning and understanding.

    Imagination involves the human capacity to organize

    mental representations into meaningful, coherent

    unities that are comprehensible.

    Johnson presents compelling evidence that embodied orimage schematic structures are metaphorically extended

    to direct and constrain peoples networks of meanings.

    * Designing to Support Reasoned Imagination through Embodied Metaphor by Alissa Antle

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    50/76

    These acts are ones of reasoned imagination

    and are used to creatively structure reasoning

    about novel concepts, perceptions and

    experiences

    Hegarty points out that Mental simulation

    may involve motor representations as well as

    visual representations.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    51/76

    Blending

    metaphor comprehension can be explained in

    terms of conceptual blending, which involves not

    mapping from a source to a target but rather

    combining and fusing two or more mental spacesin order to create a new, blended mental space,

    This blended space includes both projections

    from the constituent input domains and

    inferences not present in any of the inputs.

    The meeting Point of

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    52/76

    The meeting Point of

    Imagery, Schema, Metaphor and Blending

    The hypothesis is that subconsciously, our

    understanding of the abstract ideas like of

    economy" is that of a living entity.

    When we wish to talk about it, our

    subconscious mind invokes mental imagery on

    this living entity, allowing us to think andreason about it.

    Consider: birth of an economy and growth of economy

    *On The Computational Modeling of Metaphors thesis by Nitish, Mukerjii 2010

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    53/76

    Since this imagery uses similar neural pathways as the real

    physical imagery, the language center of the brain mightreceive similar activations, leading to same choice of wordsfor both the physical and the abstract concepts.

    In other words we can say:

    As grounding takes place In the childhood more and moreimage schemas are created in the mind, but with the age,old image schema are used to model new ideas.

    Because the brain tries to represent the new knowledgeusing minimum infrastructure*

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    54/76

    Computational Approaches

    so far to

    Image Schema and Metaphor

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    55/76

    Computational Metaphor Identification to

    Foster Critical Thinking and Creativity, Phd

    Thesis by ERIC BAUMER, UNIVERS ITY OF

    CALIFORNIA, IRVINE,

    Working on Computational MetaphorIdentification and

    To model Metaphors to understand howhuman thinks. And to

    Not well written Thesis

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    56/76

    IS Language

    Build by St.Amant et al. [2006]. It is computational formalization , called Image Schema

    Language,

    a language in which image schemas can be modeledcomputationally,

    The target of designing ISL is to let image schemas have arelational structure with compositional semantics thatadmits operations of interpretation and permits crossdomain transfer of image schema structure.

    In another words, ISL represents image schemas in ageneral purpose syntactic forms with the property thatsyntactic operations on them are equivalent to semanticoperations in an indefinitely large number of domains.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    57/76

    For each image schema, ISL represents it as anobject with internal slots and set of operations.

    Slots are used to represent the relations withother objects.

    Operations determine the capabilities of animage schema object.

    For example, a container object may take oneobject as its content, one object as its boundary.

    It also has operations like put into and getout.

    Not Much Explained Paper

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    58/76

    JEAN System

    Event, Action System, by the developer of IS

    Language

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    59/76

    Nash Equilibrium

    A fundamental concept in game theory and

    economics

    most widely used method of predicting the

    outcome of a strategic interaction/Decision

    Making

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    60/76

    Nash Equilibrium

    Ifwe all go for the blonde and block each other,not a single one of us is going to get her. So then wego for her friends, but they will all give us the cold

    shoulder because no on likes to be second choice.But what if none of us goes for the blonde? We

    won't get in each other's way and we won't insultthe other girls. It's the only way to win. It's the only

    way we all get laid [2].

    [2] A Beautiful Mind. Dir. Ron Howard. Universal Pictures, 2001

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    61/76

    Prisoner's Dilemma

    A game frequently displayed in television policedramas. Two partners in crime are separated intoseparate rooms at the police station and given a similardeal. If one implicates the other, he may go free whilethe other receives a life in prison. If neither implicatesthe other, both are given moderate sentences, and ifboth implicate the other, the sentences for both aresevere. Each player has a dominant strategy toimplicate the other, and thus in equilibrium eachreceives a harsh punishment, but both would be betteroff if each remained silent. In a repeated or iteratedprisoner's dilemma, cooperation may be sustainedthrough trigger strategies such as tit for tat.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    62/76

    Coordination game

    The coordination game is a classic (symmetric) two player, twostrategy game, with an example payoff matrix shown to the right.The players should thus coordinate, both adopting strategy A, toreceive the highest payoff; i.e., 4. If both players chose strategy Bthough, there is still a Nash equilibrium. Although each player isawarded less than optimal payoff, neither player has incentive to

    change strategy due to a reduction in the immediate payoff (from 3to 1).

    Player 2

    adopts

    strategy A

    Player 2

    adopts

    strategy BPlayer 1 adopts strategy

    A4, 4 1, 3

    Player 1 adopts strategy

    B3, 1 3, 3

    R h P l

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    63/76

    Research Proposal

    Building on: Concepts to Think

    and Reason Creatively In the proposed system , we will develop complex concepts by

    mapping simple ones into each other, to let the computer

    think and reason creatively Our system will be having some basic level ofconcepts in the

    form of simple Image Schemas about world objects like up-

    down, containers., movements/actions etc.

    Image Schema will lie at the basis of numerious metaphoricalconstructions to let them project from one domain to other

    domain.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    64/76

    That is the basic set of schema making a concept aboutsomething will be used as metaphor ( transfer basic

    idea into a new idea using conceptual metaphor) tocreate a new idea or concept.

    There might be conceptual blending in which a newconcept will be created.

    ( If I were you, I would have done that) ( Example ofConceptual Blending)

    The system will be able tothink of its owntoperceivelanguage, and reason in a controlledsituation/environment (with Basic Image Schema andother inputs/stimuli)

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    65/76

    There will be a number of basic schemas.

    All these basic schemas will be well defined in terms ofmeaning, its usage, its compositional structure, itsintegrational/compositional properties with otherschemas, i.e. what is the necessary properties of aschema to be combined with any other schema. inwhat circumstances, and

    After integration or combining with other schema what

    will be the criteria to accept the newly born schema asfamily, or part of the domain or to decide a newdomain.

    Structure of the System

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    66/76

    Following are different Basic Schemas with differentmeaning for each schema in different situations.

    Simple situation is presenting the standard scenario,

    Accepting situation presents the values/action/eventsit can receive or get from any other schema.

    Whereas provider situation shows the conditions inwhich a schema can provide some properties of itsown to any other schema to make a new compound

    schema to create more complex meaning or conceptsor perception about things or events.

    Some Rules for IS

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    67/76

    Some Rules for IS

    1Arrow

    Schema

    Stands for Simple

    Situation

    Accepting

    Situation

    Accepting Properties Provider

    Situation

    Provider

    Properties

    Marri

    age

    to

    aa Going Out With something in hand something

    in hand

    1,2,3

    aaa Going Up Over all other stuff Throwing

    something

    2

    aaaa Going Down The hill Riding onbike

    3

    aaaaaa Going Back With success in hand 2,3

    2

    Circle

    Schema

    Cc

    Ccc

    Cccc

    Ccccc

    3Force

    Schema

    Fff

    Fffff

    Fffffff

    Fffffffff

    Ffffffffffff

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    68/76

    Marriage show which Schema can be combined with whichone like, 1 with 2 and which properties it can give and taketo form a third schema,

    what will be the properties of 3 schema they will make,

    what properties the 3 schema can share and take to otherschema when it will combine with any other schema..

    what can be the situation in which a schema will combinewith another schema to produce good schema and whatcan be the situation to produce a bad or ill schema..

    what is an ill schema .what is a good schema How Nashs Equilibrium will play its role to achieve

    optimum position.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    69/76

    IS-DIS-A IS-B IS-C

    Com IS-II

    Concept

    Eq

    Eq

    G

    Eq

    G

    Eq

    G

    G

    G

    es

    t

    a

    lr

    t

    C

    yc

    l

    e

    Where the Flag might show:National Pride

    Pakistan

    A country

    An Asian Country

    Still To Work On:

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    70/76

    What Inputs to Perceive on

    Words

    ConceptsHow

    NN

    ?

    Emotions as driving force for

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    71/76

    I Always believed in numbers, in equations and logic that leadto reason, but after a lifetime of such pursuit I asked for whattruly is logic? Who decides reason? My quest has taken me to

    the physical, the metaphysical, the delusional, and back. Ihave made the most important discovery of my career - themost important discovery of my life. It is only in the

    mysterious equations of love that any logic or reason can befound. I am only here tonight because of you. You are the only

    reason I am. You are all my reasons*

    Thank You

    * A Beautiful Mind. Dir. Ron Howard. Universal Pictures, 2001

    Emotions as driving force for

    Intelligence and Image Schema

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    72/76

    Supporting Material

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    73/76

    Creativity

    creativity to be exhibited needs atleast

    domain knowledge. no doubt creativity is

    gifted property of human mind but it doesnt

    work like Miracle Creativity is the peak of intelligence, it finds

    out a way out no way using some unrelated

    relashionships among previous knowledgeand experience.

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    74/76

    Creativityany thing unusual with some value iscreativity.all human with average intelligence arecreative even with the small things they do in theirevery day lifeI go to Karachi using direct flight all golike this another go through 55 stops because he has

    some other tasks to do his way to karachi so he planshis route using 55 stops in the way.these stops neverrouted/follwed by any one else so he is creative.butcreativity is assumed something of great importancelike a great invention or a great discoverybut formachines we can try to follow the simple form ofcreativity

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    75/76

    Evolution is an imperfect and often violentprocess, a battle between what exists and whatyet to be born ( Heroes S01E05)

    For years scientist stumps by the mystery ofwhales migration. In such a big ocean how dothey they find each other. Then some scientistsrecorded their songs in the wild. They dont sing

    in captivity ( Heroes S04E08) The Universe isnt random, there are moves and

    forced moves(Alphas S01E01)

  • 8/2/2019 Machine Perception and Creativity Building on Image Schema Conceptual Metaphor and Blending by Zaheer Ahmad Dated 19-05-2011

    76/76

    Context and Culture help to understand

    natives to understand their language well as

    compare to well trained non native speakers.