amr-to-text generation - grammatical frameworkschool.grammaticalframework.org › 2017 › slides...

AMR-to-TextGenerationviaGF

NormundsGrūzītis

UniversityofLatvia,InstituteofMathematicsandComputerScienceNationalinformationagencyLETA

GFSummerSchool2017,Rīga,Latvia

This work has received funding in part from theLatvian State research programs SOPHIS andNexIT, the EU Horizon 2020 project SUMMA(grant No. 688139), and the European RegionalDevelopment Fund (grant No. 1.1.1.1/16/A/219).

• Framesemantics– FrameNet– PropBank

• AMR

• Text-to-AMRparsing,AMR-to-textgeneration– SemEval 2016– SemEval 2017

Agenda

SemanticRoleLabelling (SRL)

TurboParser +SEMAFOR:http://demo.ark.cs.cmu.edu/parseFrameNet

LTHparser:http://barbar.cs.lth.se:8081/PropBank

FrameNet (https://framenet.icsi.berkeley.edu)

FrameNet (FN)• Alexico-semanticresourcebasedonthetheoryofframe

semantics(Fillmoreetal.2003)

– Asemanticframe representsacognitive,prototypicalsituation(scenario)characterizedbyframeelements(FE)– semanticvalence

– Framesare“evoked”insentencesbytargetwords – lexicalunits(LU)

– FEsaremappedbasedonthesyntacticvalence oftheLU

• ThesyntacticvalencepatternsarederivedfromFN-annotatedcorpora(foranincreasingnumberoflanguages,incl.Latvian)

– FEsaresplitintocore andnon-core ones

• CoreFEsuniquelycharacterizetheframeandsyntacticallytend tocorrespondtoverbarguments

• Non-coreFEsarenotspecifictotheframeandtypically areadjuncts

BerkeleyFrameNetasInterlingua

want.v..6412 känna_för.vb..1

IntroducedinBFN,reusedinSweFN

e.g.“[I]Experiencer don't WANT [todeceive anyone]Event”|

anembeddedframe

SomevalencepatternsfoundinSweFNSomevalencepatternsfoundinBFN

e.g.“[Jag]Experiencer KÄNNER FÖR [en turpålandet]Focal_participant”

FrameNetandGF• ExistingFNsarenotentirelyformalandcomputational

– Alimited butcomputational FN-based GFgrammarandlexicon

• GrammaticalFramework:– Separatesbetweenanabstractsyntax andconcretesyntaxes– Providesageneral-purposeresourcegrammarlibrary (RGL)

• Thelanguage-independent layerofFrameNet(framesandFEs)–theabstractsyntax– Thelanguage-specific layers(surfacerealizationofframesandFEs;LUs)–

concretesyntaxes

• RGLcanbeusedforunifying thesyntactictypesusedindifferentFNsandfortheconcreteimplementation offrames– FrameNetallowsforabstracting overRGL

Usecase(1)• Provideasemantic APIontopofRGLtofacilitatethedevelopment

ofGFapplicationgrammars– Incombinationwiththesyntactic APIofRGL– Hidingthecomparativelycomplexconstructionofverbphrases

mkCl person (mkVP (mkVP live_V) (mkAdv in_Prep place))-- mkCl : NP -> VP -> Cl-- mkVP : V -> VP-- mkVP : VP -> Adv -> VP-- mkAdv : Prep -> NP -> Adv

Residence -- Residence : NP -> Adv -> V -> Clperson -- NP (Resident)(mkAdv in_Prep place) -- Adv (Location)live_V_Residence -- V (LU)

Usecase (2)• FN-annotatedknowledgebasesàmultilingualverbalization

Imants Ziedonis ir dzimis 1933.gada 3.maijā Slokas pagastā.Imants Ziedonis wasborn in Sloka parishon 3May1933.

• Frame valencepatternsarerepresentedbyfunctions– TakingoneormorecoreFEs(A-Z)andoneLUasarguments– ReturninganobjectoftypeClause whoselinearizationtypeis

{np: NP; vp: VP}

• FEsaredeclaredassemanticcategoriessubcategorizedbythesyntacticRGLtypes– NP,VP,Adv (includesprepositionalobjects),S (embeddedsentences),QS

FrameNet-basedgrammar:abstract

cat Event_VP cat Focal_participant_NP

cat Experiencer_NP cat Focal_participant_Adv

fun Desiring_V : Experiencer_NP -> Focal_participant_Adv -> V -> Clause

fun Desiring_V2 : Experiencer_NP -> Focal_participant_NP -> V2 -> Clause

fun Desiring_V2_Pass : Experiencer_NP -> Focal_participant_NP -> V2 -> Clause

fun Desiring_VV : Event_VP -> Experiencer_NP -> VV -> Clause

• ThemappingfromthesemanticFrameNettypestothesyntacticRGLtypes issharedforalllanguages

– LinearizationtypesareoftypeMaybetoallowforoptional(empty)FEs

• Toimplementtheframefunctions,RGLconstructors areappliedtotheargumentsdependingontheirtypesandsyntacticroles,andthevoice

FrameNet-basedgrammar:concrete

lincat Focal_participant_NP = Maybe NP

lincat Focal_participant_Adv = Maybe Adv

lin Desiring_V2 experiencer focal_participant v2 = {

np = fromMaybe NP experiencer ;

vp = mkVP v2 (fromMaybe NP focal_participant)}

lin Desiring_V2_Pass experiencer focal_participant v2 = {

np = fromMaybe NP focal_participant ;

vp = mkVP (passiveVP v2) (mkAdv by8agent_Prep (fromMaybe NP experiencer))}

http://grammaticalframework.org/framenet/

SemanticRoleLabelling (SRL)

TurboParser +SEMAFOR:http://demo.ark.cs.cmu.edu/parseFrameNet

LTHparser:http://barbar.cs.lth.se:8081/PropBank

PropBank (http://propbank.github.io)

AMR (Abstract Meaning Representation)• FromSRLtowhole-sentencemeaningrepresentation

– Incl.PropBank SRL,NERandNEL,treatmentofmodality,negation,etc.

• Simpleandcompact datastructure– PENMANnotation:directedlabeledgraphencodedinatree-likeform– Easytoreadandwrite(forahuman),andtraverse(foraprogram)– Langkilde andKnight(1998)à Banarescu etal.(2013)

• Aimedatlarge-scale humanannotationandsemanticparsing– Practical,replicableamountofabstraction– Anactualsembank of40K+sentences

• Capturesmany aspectsofmeaning– Aimstoabstract awayfrom(English)syntax

• Nodesarevariables labelledbyconcepts– Entities,events,states,properties– s / soldier:s isaninstanceofsoldier

• Edgesaresemanticrelations

• AMRabstractsinnumerouswaysbyassigningthesameconceptualstructuretodifferentsurfacerealizations

AMR (Abstract Meaning Representation)

f /fear-01

d /die-01s /soldier -

polarity

ARG1(Pustetal.,2015)

SchneiderN.,Flanigan J.,O’GormanT.AMRTutorialatNAACL2015

https://github.com/nschneid/amr-tutorial/

• AMRisstillbiasedtowardsEnglishorothersourcelanguages

• Meanwhile,AMRisagnostic abouthowtoderivemeaningsfromstrings,andviceversa

• Xue N.,Bojar O.,Hajič J.,PalmerM.,Uresova Z.,ZhangX.NotanInterlingua,butclose:ComparisonofEnglishAMRstoChineseandCzech.LREC2014

AMR (Abstract Meaning Representation)

Text-to-AMR:humanannotationhttps://amr.isi.edu/editor.html

AMR-to-text:humanevaluation

SampleAMR (1)#::snt Afourthmember,Jean-MarcRouillan,remainsbehindbars.

(r / remain-01:ARG1 (p / person

:wiki -:name (n / name

:op1 "Jean-Marc" :op2 "Rouillan"):mod (p2 / person

:ARG0-of (h / have-org-role-91:ARG2 (m / member))

:ord (o / ordinal-entity:value 4)))

:ARG3 (b / behind:op1 (b2 / bar)))








Remainingmembersperson4jean-marcrouillan – behindbar.

Jean-MarcRouillan,thatisthe4thmember,isremainedbehindabar.








Remainingmembersperson4jean-marcrouillan – behindbar. JAMR

Jean-MarcRouillan,thatisthe4thmember,isremainedbehindabar. GF








Remainingmembersperson4jean-marcrouillan – behindbar. JAMR

Jean-MarcRouillan,that isthe4thmember,isremainedbehinda bar. GF

SampleAMR (2)#::snt Theyshouldhavebeenexpelledfromschoolataminimum.

(r / recommend-01:ARG1 (e / expel-01

:ARG1 (t / they):ARG2 (s / school):degree (a / at-a-minimum)))

Shouldtheyat-a-minimumexpelschool. JAMR

Itisrecommendedthattheyareexpelledto a schoolataminimum. GF

expel-01ARG0=PAG (prototypicalagent)ARG1=PPT (prototypicalpatient)ARG2=DIR (direction)à DIR_Prepà to_Prep

ToDo:basedonstatistics fromPropBankandFrameNetcorpora,“reconstruct”Prep-s,dependingonframe/verbvalency,ARGrole,orNPhead

SampleAMR (3)#::snt Texascriminalcourtsandprosecutorsdonotcoddletoanyone.

(c / coddle-01:polarity –:ARG0 (a / and

:op1 (c2 / court:ARG0-of (c4 / criminal-03):location (s / state

:wiki "Texas":name (n / name :op1 "Texas")))

:op2 (p / person:ARG0-of (p2 / prosecute-01):location s))

:ARG1 (a2 / anyone))

Notexas texas criminalcourtandprosecutorscoddleanyone. JAMR

AcriminalcourtinTexasandapersonthatprosecutes donotcoddleanyone. GF

personthatprosecutesà prosecutororganizationthatgovernsà government

SampleAMR (4)#::snt HowLongareWeGoingtoTolerateJapan?

(t / tolerate-01:ARG0 (w / we):ARG1 (c / country

:wiki "Japan" :name (n / name :op1 "Japan"))

:duration (a / amr-unknown))

Wehavetoleratedthejapanamr-unknown. JAMR

HowlongdowetolerateJapan? GF

if ':mode expressive' in amr: amr = amr.replace(':mode expressive', ' ') + ' !'if ':mode imperative' in amr: amr = amr.replace(':mode imperative', ' ') + ' !'if ':mode interrogative' in amr: amr = amr.replace(':mode interrogative', ' ') + ' ?'if 'cause-01:ARG0(amr-unknown)' in amr: amr = 'why ' + amr.replace('cause-01:ARG0(amr-unknown)', ' ') + ' ?'if ':location(amr-unknown)' in amr: amr = 'where ' + amr.replace(':location(amr-unknown)', ' ') + ' ?'if ':ARG1(amr-unknown)' in amr: amr = 'who ' + amr.replace(':ARG1(amr-unknown)', ' ') + ' ?'if ':mod(amr-unknown)' in amr: amr = 'what ' + amr.replace(':mod(amr-unknown)', ' ') + ' ?'if ':duration(amr-unknown)' in amr: amr = 'how ' + amr.replace(':duration(amr-unknown)', ' ') + ' ?'if 'amr-unknown' in amr: amr = 'what ' + amr.replace('amr-unknown', ' ') + ' ?'

SampleAMR (5)#::snt XinhuaNewsAgency,Tokyo,September1st,byreporterYiguo Yu

(b / byline-91:ARG0 (p2 / publication

:name (n / name:op1 "Xinhua" :op2 "News" :op3 "Agency"))

:ARG1 (p / person:name (n2 / name :op1 "Yiguo" :op2 "Yu"):ARG0-of (r / report-01))

:location (c2 / city:name (n3 / name :op1 "Tokyo"))

:time (d / date-entity:month 9:day 1))

Xinhuanewsagencyhasreportedyiguo yu byline inatokyo 19. JAMR

XinhuaNewsAgencybyYiguo Yuon1SeptemberinTokyo. GF

SampleAMR (6)#::snt Alliot-MariearrivedonSunday.

(a / arrive-01:ARG1 (p / person :name (n / name :op1 "Alliot-Marie")):time (d / date-entity :weekday (s / sunday)))

Sunday'sarrivalofalliot-marie michèle_alliot-marie. JAMR

unknown qualified constant L.arrive_V2 GF

(a / arrive-01:ARG0 (p / person :name (n / name :op1 "Alliot-Marie")):time (d / date-entity :weekday (s / sunday)))

Alliot-marie michèle_alliot-marie arrivedsunday. JAMR

Alliot-MariearrivesonSunday. GF

SemEval 2017:Task9• Subtask1:ParsingBiomedicalData• Subtask2:AMR-to-EnglishGeneration

JAMR(5-grams)

AMRà(U)Dàlin

UL/IMCS/LETA

Tranducerà linSMT:AMRà Eng

Approaches:• “grammar-based”• SMT/NMT• end-to-end

SemEval

ACL

RIG-GOT-RIOà TriofromRigawithregardstoGOT&RIO;)

“We made the following resourcesavailable to participants: [..] TheJAMR (Flanigan et al., 2016)generation system, as a stronggeneration baseline. [..]” (May &Priyadarshi, 2017)

à

à

à

Underthehood

#mkVP :VV->VP->VP#(frame1 (:ARG1(var frame2)))=>(frame1 (mkVP frame2))/VV_FRAME/ < (/:ARG1/=vp < (/VAR/=var < /FRAME/=v)) #Tregex[move v >1 vp] [relabel vp /^.+$/mkVP/] [delete var] #Tsurgeon

• AlastairButler.Deterministicnaturallanguagegenerationfrommeaningrepresentationsformachinetranslation.NAACLWorkshoponSemantics-DrivenMachineTranslation,2016

InspiredbyButler (2016)

MultilingualAMR-to-Text:experimentTestTrees: t01_girls_see_a_boyTestTreesEng: a girl sees a boy .TestTreesLav: meitene redz zēnu .TestTreesRus: девочка видит мальчика .

TestTrees: t04_two_pretty_girls_see_a_boyTestTreesEng: 2 pretty girls see a boy .TestTreesLav: 2 jaukas meitenes redz zēnu .TestTreesRus: 2 хорошенькие девочки видят мальчика .

TestTrees: t21_girls_who_see_the_game_like_the_boys_who_playTestTreesEng: a girl that sees a game likes a boy that plays .TestTreesLav: meitenei , kas redz spēli , patīk zēns , kas spēlē .TestTreesRus: девочка , которая видит игру нравдит мальчика , которого играет .

TestTrees: t27_they_are_thugs_and_deserve_a_bulletTestTreesEng: they are a thug and it deserves a bullet .TestTreesLav: viņi ir slepkava , un pelna lodi .

UnderthehoodThe overall AMR-to-text process:1. The input AMR is rewritten from the PENMAN notation to the the

LISP-like bracketing tree syntax.2. In case of a multi-sentence AMR, the graph is split into two or more

graphs to be processed separately.3. For each AMR, a sequence of tree pattern-matching transformation

rules is applied (Tregex + Tsurgeon), acquiring a fully or partiallyconverted GF abstract syntax tree (AST).

4. In case of a partially converted AST, the pending subtrees arepruned.**

5. The resulting ASTs are passed to the GF interpreter for RGL-basedlinearization.

6. Since RGL supports many more languages (30+), this approach canbe extended to multilingual AMR-to-text generation, given a largetranslation lexicon (15+).

Underthehood

** Our SemEval submission:Because the coverage of our hand-crafted AMR-to-AST transformationrules is currently far from complete, we used JAMR Generator (Flanigan etal., 2016) as a “fall-back” option for AMRs that are not fully covered bythe current rule set (~200).

However, we applied heuristic post-processing rules to the JAMR output,which might have influenced the human judgements:

• Adding a full-stop, or question mark, or exclamation mark at the endof the sentence, or a wh-word at the beginning, based on the AMRconstructs.

• Removing the remaining (unresolved) AMR constructs and concepts.

• Converting large numbers into words, adding some prepositions, etc.

TheRoleofCNLandAMRinScalableAbstractiveSummarizationforMultilingualMediaMonitoring

BBCmonitoringjournaliststranslate from30languagesintoEnglish,follow 400socialmediaaccounts everyday.

Amonitoringjournalisttypically monitors 4TVchannelsandseveralonlinesourcessimultaneously.Thisisaboutthemaximumthatanypersoncancopewithmentallyandphysically.Therequiredhumaneffortthusscaleslinearly withthenumberofmonitoredsources.

Monitoringjournalistsconstantlyneedtobeonthelookoutformoresources andfollowimportantstories—butasitis,theyaretieddownwithmundane,routinemonitoringtasks.

Monitoring250videochannels resultsinadailybufferof2.5TB,aweeklybufferof19Tb,andanannualbufferof1Pb.

Large-scalemediamonitoring

Identifypeople,places,events ofinterestDiscovertrends,emergingevents,crucialnewstories

H2020grantNo.688139

SUMMA –ScalableUnderstandingofMultilingualMediA

• Event-basedmulti-documentsummarization• Storylinehighlightsacrossasetofrelatedstories

Storylinehighlights

• Extractivesummarizationselectsrepresentativesentencesfromtheinputdocuments

• Abstractivesummarizationbuildsasemanticrepresentationfromwhichasummaryisgenerated

• Whatsemanticrepresentation?

– PropBank/FrameNet– AMR

SentenceA:IsawJoe’sdog,whichwasrunninginthegarden.SentenceB:Thedogwaschasingacat.Summary:Joe’sdogwaschasingacatinthegarden.

LiuF.,Flanigan J.,ThomsonS.,Sadeh N.,SmithN.A.TowardAbstractiveSummarizationUsingSemanticRepresentations.NAACL2015

Abstractivetextsummarization

1. Riga (ULatvia,IMCS/LETA):0.6196

2. CAMR (UBrandeis/BoulderLearningInc./RensselaerPolytechnicInstitute):0.6195

3. ICL-HD(Ruprecht-Karls-Universität Heidelberg):0.6005

4. UCL+Sheffield (UniversityCollegeLondon/USheffield):0.5983

5. M2L(KyotoUniversity):0.5952

6. CMU(CarnegieMellonUniversity/UWashington):0.5636

7. CU-NLP(OKRobotGoLtd./UColorado):0.5566

8. UofR (URochester):0.4985

9. MeaningFactory (UGroningen):0.4702*

10. CLIP@UMD(UMaryland):0.4370

11. DynamicPower (NationalInstituteforJapaneseLanguageandLinguistics):0.3706*

* Rule/grammar-based;didnotuseAMRtrainingdata

SemEval 2016 Task8onAMRparsing

• Unrestrictedlarge-scaleNLU isdifficultforgrammars– SemEval 2016:few grammar-basedsystems– SemEval 2017:no grammar-basedsystems(Boxergaveup…)

• ForNLG,grammar-basedsystemsareverycompetitive!

• ScalingupAMR-to-AST:– AddmoreTregex/Tsurgeon rules– Amoreflexibleandsystematicgraph/tree-transducer(likeUD2GF)– Learningtransformationrules(C6.0;trainingdata?)– Seq-to-seq deeplearning?

Conclusion

• NormundsGrūzītis,Pēteris Paikens,Guntis Bārzdiņš.FrameNetResourceGrammarLibraryforGF.CNL2012

• DanaDannélls,NormundsGrūzītis.ExtractingabilingualsemanticgrammarfromFrameNet-annotatedcorpora.LREC2014

• DanaDannélls,NormundsGrūzītis.ControllednaturallanguagegenerationfromamultilingualFrameNet-basedgrammar.CNL2014

• NormundsGrūzītis,DanaDannélls,BenjaminLyngfelt,Aarne Ranta.Formalising theSwedishConstructiconinGrammaticalFramework.GEAF2015

• NormundsGrūzītis,Guntis Bārzdiņš.TheroleofCNLandAMRinscalableabstractivesummarizationformultilingualmediamonitoring.CNL2016

• NormundsGrūzītis,DanaDannélls.AMultilingualFrameNet-basedGrammarandLexiconforControlledNaturalLanguage.LanguageResourcesandEvaluation,51(1),2017

• NormundsGrūzītis,Didzis Goško,Guntis Bārzdiņš.RIGOTRIOatSemEval-2017Task9:CombiningMachineLearningandGrammarEngineeringforAMRParsingandGeneration.SemEval 2017

Publications

amr-to-text generation - grammatical frameworkschool.grammaticalframework.org › 2017 › slides...

Documents