PowerPoint

Activation Function40i

value of unit I in the next iteration: t+1

decay constantValue of unit i from previous iterationInput to unit ISum of products of unit j by all its input weights40 = 0.1 .Jump to the EndWe will show how:Two networks successfully simulate the time course of ambiguity resolution in the two cerebral hemispheres. This Dual Hemispheric Reading Model makes novel and surprising predictions that were borne out in a subsequent behavioral study that no other simulated model had done. Connecting both models together during learning and testing phase, computational advantage of two separate networks, successfully recovers the mistakes.22Learning Function: Changes in weights

41Learning constantDifferences in weight between units I and JSum across inputs to unit I from feedback from unit JTarget activation of unit JTarget activation of unit I41 IntroductionHuman understanding of written words required different source of information from long-term memory.

To understand the meaning of the letterswind or we use:

Lexical - prior degree of frequency or familiarity with the homographContextual prior contextual informationPhonological Do we use it? 44A.Reading is a skill, which we learn in our youth.It require us to combine different source of information,For example to understand the meaning of Ambiguous word like: " ",We use previous clues from the sentence,We use our familiarity with the meaning of this word and the obvious meaning to us,And we use a different phonological diversion of this word.B.In this research we investigated how fully recurrent connectionist network, as well as Hebrew readers process homophonic homographs versus heterophonic homographs and what is the role phonology plays in silent reading.Examples of Homographs5The businessman went to the bank.

The fisherman fished on the bank.5 , :Homophonic Homograph6She has a unique ringon her finger.on her phone.Heterophonic Homograph (rare in English)Polishthe silver.people are nice.DominantDominantSubordinateSubordinate6HebrewThings get even more complicated in Hebrew.

Most of the words written in Hebrew are omit vowels, nevertheless we understand what we read and we disambiguate words on a daily bases.

Hebrew has Homophonic and Hetrophonic Homographs in abundance.7Standard Model in Silent ReadingAlthough we have two hemispheres in our brain, both specializing in different tasks, most of the connectionist models that explain how people read do not take into account their properties.8

The Diagram is taken from Seidenberg, M.S., & McClelland J. L. Split Brain Movie (Michael Gazzaniga)Split BrainScientific American Frontiershttp://pbs-saf.virage.com/cgi-bin/visearch?user=pbs-saf&template=template.html&squery=Pieces%2Bof%2BMind

99Split Brain in HumanWhy do we need two hemispheres?What do we benefit from using both of them for the same task?10

TrainingIn training the network learns Bipolar values only(+) input+1 value(-) input-1 value

connections between the units were set to small random values.

Each training session a random homogeneous entry was presented to the network Dominant meanings were selected more than subordinate meanings

Each network learn until it successfully learn the all vocabulary.

1111 :A. 0 . B. , " .C. .D. " 2000 .E. 12 12 .

Connectionist Network12InputInputInputOutputOutputOutputOrthographyPhonologySemantic Meaning12 : 216 : . .

" . - .

Testing12 identical networks were used to simulate 12 subjects

After training the networks were first tested by presenting only orthographic inputsThe following tests included contextual clues which aided in meaning selection

We tested the net by presenting a vector that contains:+0.25 if the corresponding input feature was (+) -0.25 if the corresponding input feature was () 0 if the corresponding input feature was neutral

Output of the networks were between -1 and +1

1313 :A. ( ) .

B. : .The Split Reading Simulation Model

1414Additional results-Humans vs. Simulation15

Why Do We Need Two Hemispheres?16Example for using both hemisphere

You have a very nice ring On your IPhone.For the bullfightFor your engagement.

Results on the Model (counter clues)Dominant MeaningSubordinateMeaningNon-convergentLH88%0%12%RH17%63%20%LH + RH + Low Noise097%3%17LH only - LH receives counter clues without intervention from RH.RH only - RH receives counter clues without intervention from LH.

LH+RH +Low Noise - LH receives counter clues and RH phonologic and semantic information containing additional small random values.1718

LH only, RH only1819

LH failed recovery from dominant meaning1920

LH + RH + Low noise20The Connected ModelHow we can connect the models of the two hemispheres in a more natural way that will allows for recovering from change of meaning (as in humans) ?

yet still maintains the previous results (that successfully model the human differential response) ? 21Corpus Callosum Model22Right Hemisphere Network:PhonologyOrthographySemantic MeaningLeft Hemisphere Network:PhonologyOrthographySemantic MeaningCorpus Callosum 22Tests:First, learning stage was done while the LH and RH are disconnected. We connected them only while testing the model

Second, learning stage was done when the LH and RH are connected via the CC. This was performed in two manners: Free learning - no restriction on the CC weightsRestricted learning - the weights on the CC did change but were limited to 0.1 - 0.3 23Results - HomophonesNon-conv* LH/RHErrors *LH/RHRHLHNetwork architecture03702941.54 (4.19)40.32 (3.42)Without CC01401141.06 (2.93)39.18 (3.24)With CC: Weights fixed at 0.25(RH LH)719122340.69 (4.48)39.77 (4.21)With CC: Weights fixed at 0.25 (LH phonology RH phonology, RH semantics LH semantics).111792140.14 (4.77)39.84 (5.33)With CC: Weights fixed at 0.25 RH LH and 0.10 LH RH1933243140.79 (5.64)40.36 (6.03)With CC: Weights fixed at 0.25(All, Both ways)24

Error and Non Converge is from 96 words From 12 trials X 8 world24

Results - HomophonesFree learning RH & LH cannot perform the "Change of heart" for homophones With CC(weights restriction to 0.25).RH & LH can perform the "Change of heart". Note LH recovery is partial25Connected learning yield the following results:Free learning of CC weights caused the LH and RH to lose their special properties .Restricted learning cause the LH and RH to not lose their special properties.Results - HetrophonesNon-conv* LH/RHErrors*LH/RHRHLHNetwork architecture5011028.07 (5.14)30.39 (4.88)Without CC307027.51 (5.37)30.14 (5.11)With CC: Weights fixed at 0.25 (RH LH)9513729.23 (5.93)29.77 (6.52)With CC: Weights fixed at 0.25 (LH phonology RH phonology, RH semantics LH semantics).2416928.36 (7.20)29.63 (7.13)With CC: Weights fixed at 0.25 RH LH and 0.10 LH RH.1512181929.95 (8.67)29.32 (9.31)With CC: Weights fixed at 0.25(All, Both ways)26

Results - HetrophonesWith CC(weights restriction to 0.25)Both LH and RH can perform the "change of heart" for Heterophones The same diagram as the previous figure but presented here with standard deviation27

Connected learning vs. Separate learning Connected learning has better performance in convergence time then with separate learning.

Free learning of the CC weights causes the network to lose the "weakly coupled" proportions and therefore the LH and RH lose their special properties.

Learning with bounded weights on the CC produces the desired properties provided that the CC bounded weights are less in proportion to the interior hemispheric weights.28SummaryPresented Models of both the RH and LH, with architectural differences between the hemispheres

The hemispheres are linked together in a natural fashion, both during during learning and functioning

Connections between the hemispheres allow additional functionality for the LH as observed in humans ("change of heart"); and the hemispheres also perform at comparative speeds that also qualitatively match human DVF experiments

Our work predicts certain relative connectivity strengths between the two hemispheres in architectural regions; and thus suggests new human experiments. 2930

Thanks to

HIACS Research Center

30Thanks for listening31

31T-test= >0.01 Per sample, one sample.

32

32T-test= >0.01 Per sample, one sample.

Additional ResultsThe subordinate results are contestant with what in known from human experiments at 1000SOA.

The dominant results are contestants with what in known from human experiments although this has not been tested at 1000SOA33Additional results-Humans vs. Simulation34

*

Homophone vs. Hetrophone Words1000ms after the context was handed out, the ambiguous word was presented.The result revealed that Heterophones and Homophones were disambiguated differently in the two cerebral hemispheres in a reverse pattern The results of the computational model on subordinate meaning parallel the ones that are related to our research involving humans The results of the computational model on dominant meaning parallel the ones that are related to what we know about humans although this has not been tested at 1000SOA35Additional results-Humans vs. Simulation36

37

37T-test= >0.01 Per sample, one sample.

Learning Function: Changes in weights

38Learning constantDifferences in weight between units I and JSum across inputs and weights to unit I from feedback from unit JOutput of Target unit JOutput of Target unit Iji38 Activation Function

39value of unit I in the next iteration: t+1

decay constantValue of unit i from previous iterationSum of products of unit j by all its input weightsiInput to unit I39 = 0.1 . 43

43T-test= >0.01 Per sample, one sample.

Change from Dominant to Subordinate MeaningYou have a very nice ring on your phone.

, ,

4444Assessment 1: Access to MeaningWe measured the networks number of iterations for all the units until they became saturatedUnits in the fields of pronunciation, part of speech, and meaningWe compared the pattern of activity of the RH and LH networks response corresponding to the dominant and subordinate meaning of a given homograph across the iterations.

4545 :A.: .B. .C. , : . 50 .Previous finding 46Right HemisphereLeft HemisphereSlower and less accurateFasterLanguage ProcessingUnable to derive phonology from printAble to derive phonology from printPhonological Abilities When encountering a word:The RH activates and maintains a broader range of subordinate and distantly related meaningsWhen encountering a word:The LH quickly focuses on a narrow range of dominant and closely related meaningsFrequenciesConflicting Findings: some say the RH is more sensitive to context, and vice versaContext46ConclusionsLeft Hemisphere: All sources of information (e.g., phonology and semantics) are available immediately.As a result, selection processes are faster and more sensitive to phonological information.

Right Hemisphere: Not all sources of information are available immediately.As a result, selection processes are slower and less sensitive to phonological information.

4747RepresentationThe 288 features are grouped into sets of 16:

3 character x 16 bit 48 bit Spelling5 character x 16 bit 80 bitPronunciation2 character x 16 bit 32 bit Part of Speech8 character x 16 bit 128 bit Meaning

For example:sefernosaparno4848 " 216 . / " 12 .

4 :

""

Benefits of the ModelPsychological:Expands the traditional model to include hemispheric differences in understanding words during the reading processFurthers the understanding of Dyslectic deficiencies and enhances the ensuing methods of treatmentValidates existing and future behavioral findingsComputational:Validates assumptions regarding the organization of information in the brain4949Computational Results50RHLHUnitsHeterophoneHomophoneHeterophoneHomophone13.5713.0312.238.89Phonologic14.6014.9513.0611.49Semantic16.0216.2814.1512.20Entire vector7.68%9.72%0%0%ErrorThe Split Reading Model

51Left Hemisphere Network:PhonologyOrthographySemantic MeaningRight Hemisphere Network:PhonologyOrthographySemantic Meaning

51Detailed description of the representation of the dominant sense of "52

OrthographyPhonologyPart of SpeechMeaning52 .

.

: "" "" . , "".Learning Function: Changes in weights53

ij53 " , "" '" : "" , '" .Activation Function54

i54 " : "" Activation Function55

i55 Representation24 polarized 3-letter noun homographic pairs:

12 Homophonic12 Heterophonic

Words are represented as distributed patterns of activity over a set of simple processing units.5648 Meanings56 :A.24 .12 .12 . " 48 .

B. .TrainingIn training the net learns (+) and () values only(+) input +1 value(-) input -1 value

Phase 1:Initially, connections between the units were set to small random values

Phase 2:Each training session a random homogeneous entry was presented to the network Dominant meanings were selected more than subordinate meaningsIn total, the network underwent 1300 training sessions

12 identical networks were used to simulate 12 subjects5757 :A. 0 . B. , " .C. .D. " 2000 .E. 12 12 .

TestingAfter training the networks were first tested by presenting only orthographic inputsThe following tests included contextual clues which aided in meaning selection

We tested the net by presenting a vector that contains:+0.25 if the corresponding input feature was (+) -0.25 if the corresponding input feature was () 0 if the corresponding input feature was neutral

Output of the networks were between -1 and +1

5858 :A. ( ) .

B. : .Results on Corpus Callosum Model59LH only - LH receives counter clues without intervention from RH.RH only - RH receives counter clues without intervention from LH.

LH+RH +low noise - LH receives counter clues and RH phonologic and semantic information containing additional small random values.Neutral to SubordinateNetworkLH onlyRH onlyLH+RH+low noiseError / Non-convergence17%44.7%0%39.5%0%0%Speed of convergence40.7525.4318.159Results on Corpus Callosum Model60LH only - LH receives counter clues without intervention from RH.RH only - RH receives counter clues without intervention from LH.

LH+RH- LH receives counter clues and RH phonologic and semantic information.

LH+RH +random - LH receives counter clues and RH phonologic and semantic information containing additional small random values.ErrorsNon convergesError / Non-convergence(out of 288)Speed of convergence(Iteration)MethodLH onlyRH onlyLH+RHLH+RH+randomLH onlyRH onlyLH+RHLH+RH+randomDominant to Subordinate4912901149810040.753.925.436.6834.746.6818.16.04Subordinate to Dominant0600506005526.75.314.262.1926.485.1222.384.6760Discussion :Homophones vs. Hetrophones This work shows that:Connecting the LH and RH in a more natural way draws the same conclusions in homophonesData transfer in Homophones is more beneficial when done from RH to LHData transfer in Hetrophones can be more beneficial when done from LH to RH. Note that results are less conclusive.Connection between the hemispheres via the CC is "weakly coupled" as compared to the inner hemisphere connections

61The simulationFirst, the learning stage was done while the LH and RH are disconnected. We connected them only while testing the model Second, the learning stage was done when the LH and RH are connected via the CC. This was performed in two manners: Free learning - no restriction on the CC weightsRestricted learning - the weights on the CC did change but were limited to 0.1 - 0.3 62The simulationTesting the modelpresenting just the orthographic part plus parts of the semantic (as clues to the subordinate meaning) Each unit is influenced from: External stimuli, Previous values, Sum of the inner connected units, the inter-hemispheric connected units (The CC)

63Discussion : Homophones vs. Hetrophones Previous work showed that in the homophone case running the LH without data transfer from RH has substantially worse performance inNumber of iterations to convergenceThe ability to perform the "Change of heartTransfer of data from RH to LH in homophones yielded better performance for the LH even in cases when the RH has failed to perform the recovery64Discussion : Homophones vs. HetrophonesIn Homophones the RH has less error and non-convergence cases then LH but in the cost of convergence time In Hetrophone the LH has less error and non-convergence cases then RH but again in the cost of convergence time 65Discussion Connected learning vs. Separate learning In separate learning it is shown that the different between homophone and Hetrophones in the RH are not significant but are significant in the LH In connected learning we can see that there is an advantage to transfer data from RH to LH in homophones and help the LH recover where in Hetrophone the transfer of data from LH to RH has no significant effect Note that in Hetrophones transfer of data from RH to LH has a negative effect on the LH ability to recover

66Consequences for Human Experiments Our work suggests a refinement of these experiments to check as well the connectivity strength between hemispheres. One possible method to do this, would be to use Dynamic Causal Modelling Our prediction as indicated above is that the RH is functionally connected to the LH and vice versa but in an asymmetric manner, with:The RH being more strongly connected to LH than vice versaThe inter-hemispheric connections are relatively weak compared to the intra-hemispheric connections67++--+---------++

+--+++-+-+++--++

-+-+--+-+---+-+-

+-----+++----+++s

--+-++-+-+---+--e

+++++-+--++-+-++f

--+-++-+-+---+--e

++--+---++++---+r

+-+-+++---+-++++n

---+++--+---++-+o

-++--+++---+--+-

+-+----+-++-+-++

+-+-++--+--+++-+

++-+-----++-+++-

-+----+++-+-----

+---++--+--+++-+

+++-+-++-++--+-+

-+---+++-++--+-+